An AI framework called ContraFix resolved 92% of security vulnerabilities in benchmark testing — nearly double the semantic accuracy of the strongest competing approach.
ContraFix works by generating variants of a failing piece of code — some that still fail, some that don't — then running them in parallel through probe points to compare their runtime states. That contrast tells the system where the fix boundary actually is, rather than just where the symptoms appear. Candidate patches only get accepted after a build-and-validation pass. The system also stores successful repair patterns in a two-track skill base, one track for generating useful code variants and one for refining patches that didn't work the first time.
The numbers are notable. On SEC-Bench, ContraFix backed by GPT-5-mini hit a 91.8% average resolution rate across repeated runs. On PatchEval — a cross-language test covering Go, Python, and JavaScript — it resolved 73.8% of 225 cases. More importantly, a semantic audit found 58.2% of its patches were genuinely correct, versus 31.3% for the next-best baseline. Resolution rate and semantic correctness are different things; a patch can pass a test suite without actually fixing the underlying flaw, so the semantic gap here is the real story.
Automated vulnerability repair has long struggled with the same problem: the only evidence a system gets is failure output, not a roadmap to the fix. Most prior approaches either rely on static analysis — expensive to configure per repository — or LLM agents that can edit code but lack concrete instance-level evidence to make a confident repair decision. ContraFix's contrastive runtime approach is an attempt to generate that evidence rather than assume it exists.