A new training trick lets AI agents learn skills they could never manage on their own.
Researchers tied to Alibaba's NLP team ran into a familiar wall in agent training: reinforcement learning only works if the model occasionally succeeds, and on tasks beyond its current ability, successes are too rare for the model to learn anything. Their fix, called rollout-time intervention, steps into a training run before it fails - using the same scoring rubric that normally just grades the outcome to nudge a failing trajectory toward a successful one. Those interventions are off-policy, meaning they're not moves the model would make on its own, so the team filters out any that drift too far from what the model could plausibly produce before training on them. Across five benchmarks, agents trained this way beat standard reinforcement learning baselines and started closing the gap with bigger, more capable models.
The interesting part is what this replaces: instead of paying for more human demonstrations or distilling a bigger model down, the rubric itself becomes the teacher. That's a cheaper lever to pull, and it shifts the real bottleneck in agent training from "how do we generate successful examples" to "how good is our rubric." It also means a model can, in principle, be pushed past the ceiling it would hit if left to explore on its own - the harder problem agent RL has long struggled with.
That promise rests on the quality of the rubric, and on five benchmarks the authors chose themselves - a mediocre grading standard would presumably teach mediocre habits just as efficiently as a sharp one teaches sharp ones.