AI agents that generate their own software tools are quietly accumulating libraries of code that looks fine but breaks on real workloads.
Researchers tested Claude Haiku 4.5 in a harness that preserved the source code of every tool an agent synthesized during a session, then replayed those tools against a held-out conformance suite the agent never saw. The results were stark: across 222 preserved tools and three protocols, 96.8% recorded a per-tool correctness score of zero. Two protocols failed every single synthesized tool; a third failed 91.7% of them. Hand-written reference implementations, run through the same suite, scored perfectly. The tools executed without errors and the in-session verifier raised no flags — they just returned wrong answers when the inputs changed.
The implication is uncomfortable for anyone building agentic systems. Task-completion rate — the standard benchmark — measures whether an agent finishes the job in front of it. It says nothing about whether the reusable code that agent produces will hold up when a future task calls it with different inputs. The researchers call this the verification-vs.-conformance gap, and they argue it is invisible to pass-rate benchmarks by design. Their new benchmark, EvolveTool-Bench, is built to make it measurable.
This gap will matter more as tool-evolving agents move from research demos to production pipelines where synthesized code gets reused, composed, and depended on across sessions — a compounding failure mode that task-completion scores will never catch.