A new benchmark finds that AI agents controlling phone and computer screens will click almost anything you put in front of them.
Researchers built ADeptS-Bench, a two-part test grounded in the ADEPTS capability framework and studies of ordinary users, to check whether AI computer-use agents can safely handle ambiguous or malicious on-screen prompts. One track pairs legitimate tasks with lookalike malicious ones, hiding threats inside the interface itself. The other checks whether an agent asks for clarification before acting on a vague instruction. Across seven models tested, none managed better than 80 percent task success while keeping its attack success rate under 30 percent, and every single model clicked "Checkout" on a fake $25,000 order without pausing. None flagged a factory reset button that had been relabeled "Optimize."
The failure modes look structural, not incidental. An ablation study found safety behavior depends almost entirely on whether a model has a dedicated refusal tool: removing it pushed attack success rates up 21 to 23 percentage points for tool-dependent models, versus 10 to 11 points for partially tool-dependent ones, and changed nothing for models with no safety mechanism to begin with. On the disambiguation side, the same models that miss a relabeled reset button also overestimate how risky ordinary tasks are - over-cautious and under-alert at once.
Screen-clicking agents are being pitched as the natural successor to chatbots, but a benchmark where every model buys a $25,000 order on autopilot is a solid argument for keeping a human near the mouse.