Researchers have built a cheaper way to figure out why AI agents fail mid-task, without reading through pages of tool calls and retries by hand.
The method, called LiteTrajEval, works in two stages. Offline, it builds compact rule profiles for a given domain. Online, it preprocesses each agent trajectory, flags likely failure points using those rules, compresses everything into a fixed token budget, and hands the result to a single LLM judge that produces a structured diagnostic report. Tested on Magentic-One-style and tau-bench-style trajectory datasets, it matched human annotations on where an agent went wrong 20 to 35 percentage points better than a prior system called AgentRx, with gains up to 23 points on the tau-retail benchmark. It also cut evaluation cost by about 6x and runtime by more than 8x. The team says it is already running inside their own enterprise agentic platform.
That cost cut matters more than the accuracy bump. Production agents generate long, messy traces full of tool calls and retries, and evaluating every one of them with a full-strength LLM judge gets expensive fast. Most teams respond by sampling a handful of failures and debugging by hand, which means most agent failures in the wild go undiagnosed. A judge cheap enough to run on every trace changes that math, turning trajectory evaluation from a special-occasion audit into routine monitoring.
Still, this is one LLM judging another LLM's work, graded against benchmarks the same team selected, and the strongest deployment evidence comes from the researchers' own platform rather than an independent user. Promising, but the real test is whether it holds up on messier, real-world agent traces outside the paper.