Multi-agent AI pipelines keep burning tokens long after a run has started failing — a new research framework aims to catch that waste in real time.
Researchers published a trace-based observability system for a three-agent architecture: an orchestrator, a search agent, and an execution agent. The framework monitors running tasks for warning signs — loops, budget pressure, low information gain, and tool instability — without waiting for a final answer to reveal that something went wrong. Tested against 165 traces from the GAIA benchmark, 67 runs failed to produce a usable answer; in those warned failures, an average of 58.1% of the token budget was consumed after the first warning fired. A small 10-task pilot that acted on those warnings cut the post-warning token fraction from 63.8% to 30.4%.
Token costs are not hypothetical. Every failed agent run in a production system is a bill that arrives before you know the answer was useless. The framework's layered approach — cheap real-time signals to redirect or halt the orchestrator, slower semantic checks to assess answer quality — mirrors the circuit-breaker pattern that distributed-systems engineers have applied to service failures for years, a discipline it took the industry a decade to standardize.
Ten tasks is a slim pilot in a specific three-agent setup; how these signals hold up in the messier, tool-heavy pipelines companies are actually deploying is still an open question.