A new paper proposes making one AI model double as its own fact-checker, pitting its answers against each other to catch mistakes.
Researchers behind VeriHarness, detailed in an arXiv paper published October 2, 2026, target a specific problem: when an AI agent works through a long, multi-step task, there is often no rubric or ground-truth answer available to check its work against. Their fix samples many rollouts from the same model, then assigns it two jobs: a disagreement resolver that checks conflicting claims against outside evidence, and a consensus challenger that pressure-tests claims every rollout agrees on, hunting for requirements everyone quietly skipped. Across five long-horizon workspace benchmarks and two frontier models, Gemini 3.5 Flash and Claude Opus 4.8, VeriHarness picked better answers than other tested baselines, and using its findings to revise the final answer added 6.2 and 6.4 points of average performance, respectively, over a single rollout. The team also released about 26,000 rollouts from the experiments, a dataset it says cost more than $100,000 to generate.
This matters because AI agent products are sold on the promise that they can run unsupervised through long workflows, but nobody has a reliable way to check the output without a human reading every step. VeriHarness's core finding, that models tend to agree on wrong answers and only reveal errors when they disagree, is a quiet admission that confidence and consensus are not trustworthy signals of correctness, something agent vendors rarely advertise. If verification has to be this elaborate just to catch a model's own mistakes, that is a tacit case against taking agent output on faith.
It is also a reminder that scaling these systems is not free: a $100,000 price tag on 26,000 practice rollouts shows that teaching a model to check its own homework is its own expensive research program, not a built-in feature.