A new AI model called SWE-Critix grades whether coding agents actually finished their assigned tasks - and it's right far more often than the general-purpose models it replaces.
Researchers built a post-training pipeline that fine-tunes a 30B-parameter model, Qwen3-30B-A3B, to judge coding-agent trajectories: the long, multi-step logs of an agent editing a repository, running tests, and calling tools. Given a task and its full execution trail, the model outputs a pass-fail verdict plus a plain-language explanation of its reasoning. The team combined supervised fine-tuning with reinforcement learning, training on trajectories from a dataset called CodeForge-Preview. On a held-out test set, classification accuracy climbed from 57% for the untrained base model to 66% after supervised fine-tuning, then to 78% after the reinforcement learning stage.
Coding agents are increasingly asked to complete entire engineering tasks unsupervised, but there is rarely an automatic oracle to confirm they actually did the job. A verifier that is right 78% of the time instead of 57% is the difference between a QA layer worth trusting and one barely better than a coin flip with extra steps. The researchers are releasing the models, datasets, and training code, which matters more than the headline number - it lets other labs check the work instead of taking the result on faith.
Jumping from 57% to 78% still leaves roughly one in five verdicts wrong - a reminder that grading the grader remains the real unsolved problem in agentic coding.