AI/ llm-benchmarks · ai-evaluation · llm-judges · research

Study Finds Most LLM Benchmark Gains Are Noise

A new study finds most claimed LLM benchmark gains fall below the scoring method's own noise floor, though pairwise judging offers a partial fix.

Most year-over-year gains on LLM benchmarks may be measurement noise, not real progress.

Researchers analyzed 373,019 LLM-judge scores across benchmarks using generalizability theory, decomposing variance into system, item, judge, and interaction components. They found that pointwise rubric scoring, rating a single output on a scale, hits a hard reliability ceiling no matter how many test items you add, because judge-system interaction noise doesn't shrink with sample size. Switching to pairwise comparison, judging two outputs head-to-head in both presentation orders, cuts that noise by two orders of magnitude and pushes the ceiling to 0.986. But pairwise comparison introduces its own bias: a system shown first wins 8.6 percentage points more often than the same system shown second, a bias larger than the median improvement claimed across 53 published win-rate papers the researchers reviewed.

The measured noise floor, 0.41 to 1.24 points on a 0-5 scale, dwarfs the median reported gain of 0.28 points. On MT-Bench specifically, all 17 recovered improvements the researchers could match fell below the benchmark's own floor, and 70% of win-rate claims fell below the pairwise floor. A lot of leaderboard climbing, in other words, may not be real.

The kicker: an audit of 628 arXiv papers found fewer than one in four even state whether their evaluation was run more than once.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →