A new study asks whether an AI agent's own internal signals can predict, in real time, whether it's about to succeed or fail.
Researchers introduce two methods for reading that signal: Latent Trajectory Dynamics, which tracks how an agent's internal representations shift across a multi-turn task, and the Action Representation Probe, which predicts success from the representations formed right when the agent decides on an action. They tested both across three interactive benchmarks, Bash, SQL, and Python, and three open model families, Qwen14B, Qwen7B, and DeepSeek6.7B. In every case, the internal-representation methods beat baselines that just look at the agent's generated text or score whole output sequences. The approach adds no extra inference cost since it requires no changes to prompts and no running the same task multiple times to check consistency.
That cost question is the real story here. As agentic systems get deployed in safety-critical settings, from automated coding to database operations, knowing when to trust an agent's output matters as much as the output itself. Existing calibration tricks like self-reported confidence or multi-sample voting eat compute and still lag behind what the model's own internal state apparently already knows.
Still, this is coding-and-shell agents on mid-size open models, not the frontier systems most companies are actually deploying, so treat the generalization claim with some caution until it's tested on larger models and messier real-world tasks.