A new evaluation framework asks a blunt question about AI agents: does private data actually leak, or does it just look risky on paper?
Researchers propose CIPL (Channel Inversion for Privacy Leakage), a black-box framework that traces sensitive information through six stages - source, selection, assembly, execution, observation, and extraction - to see whether it becomes something an outside attacker can actually pull out. They tested it across memory-based, retrieval-based, and tool-using agents, plus a live case study on BrowserUse, a browser-automation agent. Results varied widely: memory-stored data leaked in nearly every case, retrieval-based leakage was often only partial, and tool-use and live-agent leakage depended heavily on what the agent could observe, how closely prompts aligned with the data channel, and provider-specific behavior. A closer semantic audit also caught attacker-useful disclosures that simple exact-text matching missed entirely.
Most privacy audits check whether sensitive data sits somewhere in an agent's memory or logs, which says little about whether an outside attacker can actually retrieve it. CIPL reframes leakage as a pipeline problem, showing that identical stored data can end up fully exposed or practically unreachable depending on how an agent processes and surfaces it. That distinction matters as companies wire LLM agents into browsers, databases, and internal tools where stored and exposed keep getting treated as the same thing.
This is a lab study, not a red-team exercise against a shipping product - but the core finding, that storage labels alone don't predict recoverability, is exactly the nuance vendor privacy claims tend to skip.