A research framework called GroundEval is exposing a structural blind spot in how AI agents are evaluated.
Published this week on arXiv, GroundEval replaces the popular "LLM-as-judge" evaluation pattern with a deterministic check of what an agent actually did — what it searched, fetched, cited, and was permitted to access. In one case study, two frontier model judges scored a plausible agent response at 0.85 or higher. The execution trace told a different story: the agent had never retrieved the artifact its answer depended on, earning a GroundEval score of 0.000. The framework targets three specific failure modes the researchers call Silence (did the agent check before claiming something was absent?), Perspective (did it reason only from evidence available at the relevant time?), and Counterfactual (did it use the correct causal chain, not just a plausible one?).
This matters because LLM judges evaluate outputs, not process — and a confident, well-formed answer looks identical whether the agent did the work or hallucinated a shortcut. As agents get deployed over real codebases, legal documents, and financial records, that gap between "sounds right" and "used the right evidence" stops being a benchmark curiosity and starts being a liability.
The uncomfortable implication here is that the evaluation tooling the industry has leaned on may be systematically blind to the exact failure mode that makes agentic AI dangerous in production. GroundEval produces per-question diagnostics that pair tool activity with the agent's own narration — which is useful, though it also means evaluation now requires access to full execution traces, not just final answers. That is a higher bar than most current pipelines are built for.