A new paper argues the fix for flaky AI agents isn't better prompts - it's better testing.
Researchers make the case that risk-free deployment of LLM-based agents requires close scrutiny of their trajectories - the full record of reasoning steps, tool calls, and observations an agent produces while working. The paper catalogs why testing these systems is hard: there is no reliable oracle to judge correct behavior, agents behave non-deterministically, trajectories are difficult to validate, and the field lacks adequacy metrics to say when testing is enough. From there it moves to debugging, covering automated failure attribution, repair, and agents that revise their own behavior. The authors compile this into a deployment-readiness checklist meant to span an agent's full lifecycle.
Most agent failures never show up in the final output - they hide in the steps along the way, which is exactly where most teams aren't looking. As companies wire LLM agents into real business processes, that gap between looking fine and actually working becomes a compliance and security problem, not just a quality one.
The paper is short on solutions and long on open questions - root-cause attribution over long agent runs and trusting agents that rewrite themselves are both flagged as unsolved, a reminder that agent tooling is still catching up to agent hype.