AI writes plenty of buggy code, and now there's a pipeline built to catch and fix some of it automatically.
Researchers built an automated pipeline that generates Python code from a benchmark called LLMSecEval, then scans it with two static analysis tools, CodeQL and Bandit, alongside a separate Code Validator LLM. Findings get enriched with known attack techniques from MITRE's ATT&CK framework and common vulnerability patterns from the CWE catalog, before a second LLM generates a fix that gets re-scanned to check the result. The team tested two versions of the pipeline (one using only the validator's own findings, the other also fed the raw analyzer results) across four Claude models: Opus 4.8, Sonnet 4.6, Sonnet 5, and Haiku 4.5. The richer version beat the simpler one for every model, cutting flagged issues by 29 to 69 percent depending on which model wrote the code.
The more interesting finding is who won the fix-it contest. Opus 4.8, the strongest code generator in the test, was not the best at getting cleaned up; Sonnet 4.6 ended up with the fewest leftover vulnerabilities and the highest pass rate after remediation. That matters as more teams bolt automated security fixers onto AI coding assistants: picking a model for raw coding skill and picking one for its post-fix safety record are two different exercises.
The fixes weren't free, either: remediation introduced at least one new vulnerability in 15 to 22 percent of cases, a reminder that patching AI-written code with more AI still needs a human check before it ships.