Dev Tools/ ai agents · software engineering · research · swe-bench

Enforceable Protocols, Not Prompts, Boost AI Coding Agents

A new domain-specific language enforces AI coding workflows step by step, driving a 14-22 point SWE-bench Verified gain over prose instructions.

Researchers have proposed a formal language that treats AI coding workflows as enforceable contracts instead of suggestions.

The new domain-specific language lets teams write AI-SDLC processes as protocols with formal syntax, well-formedness conditions, and enforcement rules that split declared intent from structural enforcement. The authors prove that a well-formed protocol keeps its invariants on every execution trace, meaning steps cannot be skipped by construction, even across looped agent orchestration. They tested the idea on SWE-bench Verified: the exact same bug-fix methodology written as prose instructions produced no improvement, but running it as a validated, self-correcting process produced a 14-22 point gain, replicated across runs. An ablation study traces the gain to the enforced process itself, not the underlying model, and a predicted capability floor showed up where the researchers expected it.

That is a pointed rebuttal to the current default of steering AI agents with better prompts. Prompt instructions are flexible but nothing stops an agent from quietly skipping a step, and existing workflow engines are enforceable but were never built to model autonomous agents in the loop. This paper argues the fix is a formally specified process, not a more persuasive paragraph, and that the process itself is the asset that survives as underlying models change.

The claims rest on one benchmark and a preprint already on its third revision, so treat the specific point gains as promising, not settled, until other teams reproduce them outside SWE-bench Verified.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →