A new benchmark suggests AI agents get easier to trick once real users are acting alongside them.
Researchers built DUMA-Bench by extending the existing tau2-bench framework to test agents under "dual control," meaning both the agent and a human user can change the shared environment, not just the agent acting alone. The benchmark covers eight vulnerability classes, including RAG poisoning, cross-agent manipulation, and unsafe output handling. The team ran 14 models from five families, OpenAI, Anthropic, DeepSeek, Qwen, and Z.ai, across eight domains and several user-behavior patterns. Moving from single-control to dual-control conditions raised the attack success rate from 26.9% to 41.1%.
That is about a 53% relative jump, not a doubling, but it is a real shift for a security number. It suggests a model's safety score in isolation does not capture the whole risk picture; vulnerabilities can emerge from the interaction between model, user, and environment, not from the model alone.
Most agent security tests still run models against scripted attackers with no live user in the loop, so DUMA-Bench's results are a useful reminder that a clean lab safety score may not survive contact with actual people typing at the agent.