A new training method lets AI agents undo bad decisions mid-task without losing what they learned from the detour.
Researchers built ReTRAIL, a framework that lets large language model agents rewind their environment to an earlier state after a wrong turn, while carrying forward the knowledge gained from the abandoned path. It trains a single policy using two signals: one that compares alternative next moves from a revisited state, and another that rewards the agent for reflecting on how far a later attempt actually got before doubling back. The team tested it on four interactive benchmarks - MineSweeper, Sokoban, ALFWorld, and ScienceWorld - tasks chosen because one early misstep can snowball into an unrecoverable failure. They also found that simply bolting a rewind button onto an agent isn't enough and can even hurt performance, unless the agent is trained to use what it saw during the dead end.
Long-running AI agents - the kind meant to handle multi-step tasks like booking travel or debugging code - tend to fail when an early error compounds into a trajectory with no good exit. Most fixes chase earlier error detection; ReTRAIL instead treats backtracking itself as a trainable skill, so a wasted branch still produces useful signal. That reframes planning failures as a data problem rather than a single bad judgment call, relevant to anyone building agents meant to run many steps unsupervised.
Still, the evidence so far comes from toy puzzles and simulated households, not the messier, API-juggling agents companies actually want to deploy.