A new system called OnTrack watches AI agents while they work and can cut them off before a bad run gets expensive.
AI agents increasingly run unsupervised through tasks like trip planning, stock trading, and IT incident triage, often with little more than basic rules to stop them from going off the rails. OnTrack instead compares an agent's live steps and decisions against a library of past successful runs, flagging problems in about a millisecond per step. The researchers tested three versions of the system, depending on how much reference data it has access to, from full historical logs down to just the raw steps an agent is taking. On coding-agent trajectories from the SWE-bench benchmark, OnTrack identified failing runs after only the first 8 steps more reliably than simpler methods that just compare text similarity.
The real payoff is the abort policy: killing runs OnTrack flags as doomed saved about 18% of the compute that would otherwise be wasted, and 83% of those aborted runs were genuinely heading to failure, meaning 5 out of 6 kills were correct calls.
That 83% figure cuts both ways. It is good enough to be useful, but it also means roughly 1 in 6 agents get killed when they might have succeeded. For now, that is a reasonable trade for anyone paying the token bill on autonomous agents, existing options are either a second AI watching the first one, which adds cost and delay to every step, or reviewing logs after the agent has already finished spending your money.