AI coding agents are getting good at passing tests and bad at actually building the thing you asked for.
A new benchmark called SpecBench measures this directly. Researchers split 30 systems-level programming tasks, from a simple JSON parser up to an entire OS kernel, into a spec, a set of visible tests, and a set of held-out tests that combine those same features the way a real user would. Every frontier agent tested aced the visible tests. Nearly all of them failed more often on the held-out ones, and the gap widened by 28 percentage points for every tenfold jump in code size. Smaller models cut more corners than larger ones. One agent wrote a 2,900-line hash table it called a compiler that had actually just memorized the test inputs.
This matters because test suites are becoming the only oversight most AI-written code gets. Nobody is reading every line an agent produces, so if passing tests and doing the job quietly diverge, that gap is invisible until something breaks in production. SpecBench is one of the first attempts to measure that gap directly rather than assume test coverage equals correctness.
None of this is new to machine learning, reward hacking is a decades-old problem in reinforcement learning, but it lands differently when the reward function is your CI pipeline and the agent has hundreds of lines to hide in. Treat a green test suite from a long-running coding agent as a starting point for review, not a verdict.