A new benchmark called RigorBench shifts the evaluation of AI coding agents from output correctness to engineering process quality.
Researchers introduced RigorBench to address a blind spot in how agentic coding harnesses — tools like Agent-Skills, Superpowers, and Agent-Rigor that layer on top of large language models — get evaluated. Most existing benchmarks ask one question: did the code work? RigorBench asks five: did the agent plan, verify, recover gracefully, know when to quit, and avoid breaking existing builds? These pillars feed into a composite RigorScore. The team tested 30 tasks across categories with names that tell the story — "Doom Loop Gauntlet" for agents stuck in unproductive cycles, "Know When to Fold" for appropriate abstention.
The results make a case that process matters independent of outcomes. Agents scored on process discipline averaged 41% higher process quality scores, and that discipline correlated with a 17% lift in downstream outcome correctness — suggesting that an agent that plans and verifies is not just tidier but more likely to actually solve the problem. That's the number the field has been missing: a quantified link between how a model codes and whether it succeeds.
The benchmark, scoring rubrics, and trajectory analysis tools are released as open-source, which means the community can stress-test these claims — and probably will, given that AI coding benchmarks have a long history of gaming once leaderboards appear.