A new paper finds that redundancy math for multi-agent AI systems is broken, and throwing more data at the fix makes it worse, not better.
Researchers ran a preregistered evaluation of 18,000 missions using two instances of the same model in a two-agent handoff, with outcomes scored by deterministic code rather than an LLM judge. When one agent failed, the other failed too on 90 percent of those missions, a correlation the paper calls far too strong to treat the two as independent. Substituting a different model weakened that correlation in all six tested contrasts. But once the model already differed, further substituting a different vendor made no additional difference, a result the researchers had specifically predicted and confirmed as a null.
That matters because standard reliability estimates for multi-agent systems multiply each component's failure rate together, a shortcut that only works if failures are independent. This data says they are not, at least not when agents share a model, so operators who stack redundant agents for safety margin may be crediting reliability that is not really there. The paper's proposed fix, a finite-sample certificate built from a linear program over measured co-execution statistics, raises the certified reliability floor from 0.2455 to 0.4116 once the underlying analysis is expanded from ten to fourteen measured statistics, but only by admitting how little independence can be assumed in the first place.
The sharper finding is methodological: fitting a statistical model to the dependence and bootstrapping a confidence interval gets worse, not better, as more missions are logged, with no obvious warning sign. It is a reminder that adding more agents for redundancy is an engineering instinct borrowed from hardware, and this paper argues software agents built from the same weights do not fail like independent disks.