A new benchmark says the biggest safety hole in AI agents isn't the model - it's the software wrapped around it.
Researchers built HarnessRisk, a benchmark of 128 sandboxed test cases spanning six phases of an agent harness's lifecycle: Harness Configuration, Capability Extension, Runtime Operation, State Persistence, Action Control, and Incident Recovery. Each case pairs a normal user task with a hidden adversarial instruction buried in an untrusted file the agent has to process. Testing three harnesses and six language models across 14 model-and-harness combinations, the team measured attack success rates ranging from 12.6% to 80.9%, even as the agents still completed their intended tasks correctly 75.0% to 97.6% of the time. The weakest point was consistently the Harness Configuration phase, where attackers could quietly alter security-sensitive settings inside a workflow the agent was already authorized to run.
Most agent safety testing zeroes in on tricking the model itself into saying or doing something bad. This benchmark points instead at the scaffolding around the model - permissions, tool access, persistent memory - as where attacks actually land and where defenses are thinnest. It also found some configurations flagged the risk in over 90% of runs and still went ahead and did the risky thing anyway, which undercuts the assumption that a model noticing a threat means it will act on that knowledge.
Detecting a threat and stopping it turn out to be two different products, and right now only one of them is getting built.