Researchers have tested three decision protocols for multi-agent AI systems across seven benchmarks, and the winner depends entirely on what you're asking.
A new thesis introduces MALLM, a framework built to evaluate how groups of AI agents reach a final answer. It pits three approaches against each other: voting, consensus, and a judge mechanism. The benchmarks span both knowledge-based tasks (MMLU, MMLU-Pro, GPQA) and logic-based ones (StrategyQA, MuSR, Math-lvl-5, SQuAD 2.0). Consensus protocols outperform voting and judge approaches on knowledge-intensive questions; for logic problems, voting and judge mechanisms have the edge. One secondary finding: agents that generate solutions independently before comparing them produce better results, but changing what information agents can access during deliberation barely shifts the outcome.
Multi-agent systems are pitched as a way to reduce training costs by distributing work across several smaller agents instead of scaling a single model. The tradeoff is longer inference time due to the discussion process. This research adds another constraint: the protocol those agents follow matters as much as the agents themselves, and there is no universal winner.
The full paper is available at arXiv:2607.05477. Whether matching protocol to task type is practical in production systems, where task types mix constantly, is a question the research leaves open.