A new benchmark puts an inconvenient number on AI-generated code: even the strongest models clear a joint security-and-correctness bar less than 15% of the time.
Researchers built DualGauge, a framework that evaluates LLM-generated code against both functional and security tests drawn from the same original specification. The benchmark covers 307 tasks across Python, C++, and JavaScript. They ran 10 LLMs through it and also tested three agentic coding systems: Codex, OpenHands, and Claude Code. None of the models broke the 15% joint-success threshold in any language, and none of the usual levers moved that number reliably: not scale, not extended thinking, not instruction tuning, not quantization.
The study's sharpest finding is what it says about how the industry measures AI coding tools. Functional correctness alone, which is what most benchmarks track, substantially overstates how reliable these tools actually are. The result for agentic systems is equally deflating: iterative scaffolding provided no advantage over direct generation on these tasks, which cuts against the marketing case for coding agents as a qualitative step up from raw LLM output.
Most failures traced to two patterns: guards that exist in the generated code but don't go far enough, and mishandled output boundaries. Those are exactly the problems that pass a quick code review, and that's the point.