AI/ llm · multi-model ai · ai research · inference

COMED Teaches AI Systems When to Ask for a Second Opinion

A new controller called COMED decides when an AI model needs a second opinion, boosting accuracy without asking every time.

A new academic system decides, one query at a time, whether an AI model should phone a friend before answering.

Researchers behind COMED (Controlled Model Escalation for Multi-LLM Deliberation) built a controller that sits after a large language model gives its first answer. It checks the model's own consistency, how close the routing decision was, and a quick peer probe, then picks one of three paths: accept the answer, verify it, or escalate to a second model for full collaboration. That is a middle ground between two existing approaches - routers that pick one model and stop, and dense collaboration systems that consult every peer on every query regardless of need. Tested across 16 open-weight model setups on medical, scientific, and general reasoning benchmarks, COMED beat both approaches, with gains up to 10.7 percentage points on MedQA, while calling fewer models and using fewer tokens than dense collaboration. On frontier models, it lifted GPT-5.5's score on the HLE benchmark from 23.1% to 28.1%.

The interesting part is the paper's own admission that collaboration is not automatically good. Peer input can rescue a wrong answer, but it can just as easily talk a model out of a correct one - the researchers call this a rescue-harm trade-off. That framing matters more than the benchmark numbers: most multi-agent AI pitches assume more consultation is always better, when it can just as easily add noise and cost.

Still, this is a benchmark result from open research, not a shipped product feature. Whether assistants or coding agents adopt this kind of selective escalation - instead of the current fashion for stacking more agents and more tokens at every query - remains to be seen.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →