Getting AI models to pool knowledge fairly turns out to be a game-theory problem.
A new paper from arXiv introduces WALLA — Wagering mechanisms for LLM Aggregation — a framework for combining predictions from multiple large language models without requiring any model to reveal its private data or internal workings. Each model submits a prediction alongside a learned wager, and the system uses those wagers as aggregation weights. The mechanism's key trick is a leave-one-out baseline: a model only profits when its prediction outperforms what the group would have said without it. That structure, the authors argue, makes honest reporting the dominant strategy — models have no incentive to game the system.
This matters because multi-model ensembles are increasingly common in production AI, but most aggregation methods assume a central coordinator with access to each model's internals. That assumption breaks down when models belong to different organizations, run on proprietary data, or need to protect user privacy. WALLA offers a path to ensemble accuracy without that trust requirement, which is a genuinely hard problem to solve cleanly.
In benchmarks spanning question-answering and forecasting tasks, WALLA matched the predictive performance of centralized aggregation methods — meaning organizations aren't paying a quality penalty for the privacy guarantee. The framework also produces uncertainty-aware outputs, since a model that wagers low is effectively flagging its own low confidence.
Ensemble methods have a long history of outperforming single models, but making them work across organizational boundaries and strategic agents is where the research has lagged. Applying wagering theory to this problem is a tidy reframe — though whether it survives contact with real-world model providers who have reasons beyond accuracy to misreport remains an open question.