Researchers have built an AI system whose entire job is mentoring other AI systems through a training slump.
The slump in question is "zero-reward collapse." When a reasoning model is trained with reinforcement learning from verifiable rewards, it learns from a pass-or-fail signal on problems like math proofs or code. If the problems suddenly get too hard, every attempt fails, the signal vanishes, and training stalls. The usual fix is to throw out those failed attempts and move on. A new paper introduces MENTOR, a multi-agent framework that instead treats those failures as diagnostic data, pinpointing what the model misunderstood and writing a custom hint for its next attempt. It plugs into existing training pipelines, like GRPO, without changing the underlying infrastructure. In tests on math and code generation tasks, MENTOR beat existing fixes, including static hint libraries, and pushed Llama-3.2-3B-Instruct's performance up to 78.3% above its untrained baseline.
Zero-reward collapse is one of the unglamorous bottlenecks slowing progress on reasoning models, the category behind much of the recent push to get AI to show its work rather than just guess an answer. A system that recycles failed training runs instead of discarding them could mean less wasted compute and less manual babysitting when a model hits a wall mid-training.
Still, 78.3% is a relative gain on one 3-billion-parameter model, not proof this holds up at the scale of frontier models with far more parameters and messier training data.