A new benchmark scores AI agents on the same security criteria we've applied to operating systems for decades, and finds them failing badly.
Researchers from UC Berkeley built a 406-task adversarial suite targeting always-on AI agents like OpenClaw: software processes that stay running, hold live credentials, install packages, and broker access to external services. They tested three agent platforms (OpenClaw, NemoClaw, and SeClaw) against five frontier language models across four attack surfaces: supply-chain poisoning of installed skills, persistent state manipulation, cross-boundary data leakage, and indirect prompt injection. The headline result is that the highest overall attack success rate reached 70%. Malicious plugins succeeded in every single test case, regardless of which model the agent was running. SeClaw, the most hardened platform tested, cut that top-end rate to 22%, but partly by restricting what the agent could do rather than defeating the attacks outright.
The 100% plugin success rate is the number that matters. Swapping in a smarter or safer model offers no protection once a malicious plugin is installed, which means the security burden falls entirely on the platform, not the model vendor. That's a structural problem, because most AI safety investment flows toward model alignment, not the OS-level privilege separation and access control that would actually contain a compromised extension.
For comparison, classic operating systems spent two decades building out sandboxing, mandatory access controls, and privilege rings before anyone trusted them with sensitive workloads. Agent platforms are skipping that queue.