Five AI models graded each other's medical reasoning, and the peer review beat both solo answers and simple majority votes.
Researchers tested five AI models (Llama-3.1-8B, Qwen2.5-7B, Phi-4, DeepSeek-LLM-7B, and GPT-oss-20B) on three medical exam benchmarks: HeadQA, MedQA-USMLE, and PubMedQA. Each model first wrote out its own chain-of-thought reasoning and picked an answer. Then the models switched roles, reviewing each other's reasoning for factual errors and logical gaps, with the highest-rated chain winning. The best combination reached 82.0% average accuracy, versus 77.7% for the strongest single model and up to 78.9% for majority voting.
That gap matters because majority voting, the usual trick for squeezing more accuracy out of multiple AI outputs, just counts agreement - it doesn't check whether the agreed-upon reasoning actually holds up. This method grades the logic itself, and the researchers found the peer reviews reliably told strong reasoning from weak, with the advantage growing as more models joined the panel.
It's a sharper idea than another leaderboard screenshot, but it's still a benchmark exercise - five models arguing over multiple-choice exam questions, not a clinician using any of this. Running every question past five models for both solving and reviewing also multiplies the compute cost for a single answer, a tradeoff the paper doesn't weigh against the accuracy gain.