[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-benchmark-finds-ai-agent-attacks-jump-53-with-real-users":10,"sections":41},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":36,"feedback":40,"feedback_at":22,"cost_usd":40,"total_tokens":40},7297,"benchmark-finds-ai-agent-attacks-jump-53-with-real-users","Benchmark Finds AI Agent Attacks Jump 53% With Real Users","A new benchmark called DUMA-Bench shows attack success rates against LLM agents rise from 26.9% to 41.1%, about a 53% jump, once real users can act too.","A new benchmark suggests AI agents get easier to trick once real users are acting alongside them.\n\nResearchers built DUMA-Bench by extending the existing tau2-bench framework to test agents under \"dual control,\" meaning both the agent and a human user can change the shared environment, not just the agent acting alone. The benchmark covers eight vulnerability classes, including RAG poisoning, cross-agent manipulation, and unsafe output handling. The team ran 14 models from five families, OpenAI, Anthropic, DeepSeek, Qwen, and Z.ai, across eight domains and several user-behavior patterns. Moving from single-control to dual-control conditions raised the attack success rate from 26.9% to 41.1%.\n\nThat is about a 53% relative jump, not a doubling, but it is a real shift for a security number. It suggests a model's safety score in isolation does not capture the whole risk picture; vulnerabilities can emerge from the interaction between model, user, and environment, not from the model alone.\n\nMost agent security tests still run models against scripted attackers with no live user in the loop, so DUMA-Bench's results are a useful reminder that a clean lab safety score may not survive contact with actual people typing at the agent.","[\"ai-security\",\"llm-agents\",\"benchmarks\",\"ai-safety\"]","2026-09-23T04:00:00.000Z","2026-09-23T05:55:59.609Z","2026-09-23T05:56:05.414Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The dek claims attacks 'nearly doubles' but the actual data shows a 26.9% to 41.1% increase, which is about a 53% relative increase (roughly 1.5x), not double — fix the dek (and any similar framing) to accurately reflect the magnitude of the increase.","resolved","ai",[32,33,34,35],"ai-security","llm-agents","benchmarks","ai-safety",[37],{"name":38,"url":39},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.24662",0,{"sections":42},[43,46,50,55,60,65,69,74,79,84,89,94,99,104],{"name":44,"slug":30,"count":45,"latest_published_at":18},"AI",4264,{"name":47,"slug":48,"count":49,"latest_published_at":18},"Security","security",707,{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",369,"2026-09-23T02:13:52.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Deals","deals",202,"2026-09-22T23:00:04.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Hardware","hardware",168,"2026-09-22T23:56:03.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":18},"Science","science",133,{"name":70,"slug":71,"count":72,"latest_published_at":73},"Consumer Tech","consumer-tech",110,"2026-09-22T20:00:00.000Z",{"name":75,"slug":76,"count":77,"latest_published_at":78},"Software","software",80,"2026-09-22T23:32:52.000Z",{"name":80,"slug":81,"count":82,"latest_published_at":83},"Dev Tools","dev-tools",79,"2026-09-22T22:21:13.000Z",{"name":85,"slug":86,"count":87,"latest_published_at":88},"Startups","startups",65,"2026-09-22T22:06:48.000Z",{"name":90,"slug":91,"count":92,"latest_published_at":93},"Gaming","gaming",45,"2026-09-22T15:35:06.000Z",{"name":95,"slug":96,"count":97,"latest_published_at":98},"General","general",43,"2026-09-21T23:48:56.000Z",{"name":100,"slug":101,"count":102,"latest_published_at":103},"Reviews","reviews",27,"2026-09-22T13:00:00.000Z",{"name":105,"slug":106,"count":107,"latest_published_at":108},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]