AI/ ai · formal-verification · code-generation · benchmarks

AxDafny Pushes Verified Code Generation to 92.7%

A verifier-guided repair framework beats prior Dafny benchmarks by 6.5 points, using a model the paper names GPT-5.5 as its baseline.

A new agentic framework called AxDafny hits 92.7% verification success on DafnyBench, outpacing the previous best proof-hint baseline by 6.5 percentage points.

Researchers built AxDafny as a repair loop: a model generates code, then iteratively patches the formal proof artifacts — invariants, assertions, termination arguments — until a verifier signs off or gives up. To stress-test it, they also produced a new benchmark, LiveCodeBench-Pro-Dafny, which translates 250 competition-style programming problems into Dafny with formal specifications and a verifier-based evaluation harness. The paper identifies GPT-5.5 as the baseline model — that name is taken directly from the arXiv abstract and has not been independently confirmed as a public release. On both benchmarks, AxDafny substantially improves over that baseline.

The result matters because Dafny is one of the few languages where a compiler can actually prove code correct, not just test it — and getting AI to produce those proofs reliably has been a hard open problem. A 92.7% success rate, if it holds on independent evaluation, is the kind of number that makes formal verification look less like a research curiosity and more like something teams might actually ship.

The paper's own caveat is worth keeping: verification success and runtime test performance measure different things, which is a polite way of saying a formally verified program can still be wrong in ways the spec didn't anticipate.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →