AI/ ai · computational-argumentation · policy · llm

AI Debate Tool Lets Personas Argue Policy With Humans

A new system pairs LLM-simulated stakeholders with formal argumentation rules to keep AI-assisted policy debates from collapsing into agreement.

Researchers have built a system that lets you argue policy with AI personas designed specifically not to agree with you.

The tool, called OmouAI, combines large language models with computational argumentation, a field focused on formally representing and evaluating debates. A human user deliberates a policy claim alongside simulated personas, such as stakeholders, domain experts, or devil's advocates. Each persona generates its own arguments, and all of them, human included, feed into a shared argumentation framework that the user can contest, add to, or revise. The arguments are then scored using deterministic argumentative semantics against external benchmarks like the UN Sustainable Development Goals, with the resulting shift in those goals used as a stand-in for how good the policy recommendation actually is.

The real target here is sycophancy, the well-documented tendency of chatbots to tell users what they want to hear. That's a nuisance in casual chat. In a public-policy tool meant to pressure-test ideas, it's a design flaw that undermines the entire premise. By forcing arguments through a fixed evaluation framework instead of letting the model's own judgment score the debate, OmouAI is betting that structure, not a better prompt, is what fixes faithfulness.

Whether it holds up outside a research paper is the open question. Deterministic scoring only works as well as the framework it's built on, and mapping fuzzy political tradeoffs onto SDG indicators is its own can of worms.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →