A research team is pitching verification — judging whether an AI solution is correct — as the next axis for scaling LLMs, and their framework backs the claim with numbers across four benchmarks.
LLM-as-a-Verifier skips the usual approach of asking a model for a discrete score. Instead, it computes a continuous score from the distribution of scoring-token probabilities. That probabilistic foundation lets the framework scale along three levers: how fine-grained the score is, how many times the evaluation repeats, and how many sub-criteria the judgment is broken into. The result, the authors say, is better separation between good and bad solutions and more calibrated comparisons — without touching model weights. The team also built a cost-efficient ranking algorithm on top for picking the best candidate from a set.
The benchmark results are the headline: 86.5% on Terminal-Bench V2, 78.2% on SWE-Bench Verified, 87.4% on RoboRewardBench, and 73.3% on MedAgentBench — state-of-the-art on all four as of publication. Terminal-Bench V2 and RoboRewardBench are established evaluation suites for terminal-based agents and robotic reward modeling respectively; MedAgentBench covers medical reasoning. The framework also ships as a Claude Code extension, so developers can plug it into their own agentic pipelines to monitor task progress in real time.
Most scaling research chases bigger models or more training data — this paper argues the verifier deserves its own scaling budget. If verification accuracy keeps improving as you throw more compute at it, it changes how teams should think about inference-time spending. That said, benchmark leadership is a crowded, short-lived title in AI research: treat these numbers as a snapshot, not a moat.