AI agents can run your data pipeline flawlessly and still give you the wrong answer.
Researchers ran a cross-domain study covering 236 analytical tasks across finance, human resources, and public safety. Despite successful workflow generation and execution in every case, they logged 153 recurring failures. The culprit was not broken code — it was missing context. Database schemas told the agent what columns existed; they did not tell it what the numbers meant. The study identifies five failure classes: comparative grounding, process reasoning, quantitative reasoning, role confusion, and policy grounding.
This matters because the industry has spent enormous effort on getting LLMs to write correct queries. The study suggests that is the easier half of the problem. The harder half is whether the agent understands the analytical concept behind the request — and right now, that understanding lives in the user's head, not in the database.
The findings echo longstanding complaints about business intelligence tools: a technically correct query against the wrong definition of "active customer" or "incident rate" is just a confident mistake. Until agentic systems can ingest richer semantic context — business rules, policy definitions, domain conventions — they will keep producing outputs that look finished and are quietly wrong.