A new benchmark called ToolPrivacyBench exposes a gap that existing AI agent evaluations mostly ignore: whether an agent overshares private information while completing a task, not just whether it finishes the task correctly.
Researchers built ToolPrivacyBench around 2,150 test cases — 1,150 fully synthetic privacy-sensitive business workflows and 1,000 adapted from existing multi-tool benchmarks. Each case comes with a policy knowledge base that defines what information each tool is allowed to receive. After an agent runs against mock business backends, an evaluator checks the recorded tool arguments and backend audit logs against those policies. Nine widely used agents were tested. The central finding: completing a task successfully does not mean an agent handled private data appropriately. An agent can finish a workflow while quietly passing unnecessary personal or business information through intermediate tool calls that never appear in the final output.
This matters because the dominant way to evaluate AI agents — did it call the right API with the right arguments? — says nothing about what extra data got transmitted along the way. As LLMs take on more agentic roles inside enterprise software, the attack surface for incidental data leakage grows with every tool added to the chain. A benchmark that audits the full execution trajectory, not just the endpoint, is a more honest measure of production readiness.
ToolPrivacyBench formalizes what it calls a need-to-know disclosure boundary — a principle that exists in data governance and security but has not been systematically applied to agent evaluation until now. It is worth noting this is a research benchmark, not a deployed audit tool; whether enterprise teams adopt it or roll their own is an open question.