The benchmark that tells you whether an AI agent was compromised says nothing about what the agent did next.
A new paper out of arXiv proposes replacing the standard binary attack-success rate with a seven-level severity scale, running from L0 to L6. The scale scores what an agent actually did after being compromised — whether the action was reversible, whether it reached outside its original scope, and whether it expanded the agent's privileges. The researchers computed scores two ways: a deterministic oracle that reads the agent's tool-call log, and a panel of three frontier language-model judges reading a sanitized version of the same log. The judge panel matched the oracle closely, with a Krippendorff's alpha of 0.91, though both shared a notable blind spot: neither reliably caught multi-step privilege escalation chains.
The practical stakes are higher than they look. The paper tested four victim models and two defenses on the AgentDojo benchmark suite and found three cases where the binary metric gave a clean bill of health that the severity scale contradicted — including one defense that posted a zero attack-success rate while still allowing an externally visible data leak through an unfiltered tool call. A defense that scores zero on attack-success rate is a very different thing from a defense that actually stops harm.
This lands at a moment when AI agents are moving from demos into production, handling email, running code, and calling APIs on behalf of real users. The red-teaming field has largely borrowed its vocabulary from static model evals, where a refusal or a jailbreak is a clean binary. Agents acting in the world don't work that way. All code, prompts, and per-episode logs from the paper are released publicly — which at least gives defenders something concrete to work with, even if the benchmark industry takes a while to catch up.