A new reinforcement learning recipe targets the exact blind spot where today's methods let AI models hallucinate with total confidence.
Researchers behind a method called DEEPO, short for Dual-Entropy Enhanced Policy Optimization, say the reinforcement learning used to sharpen reasoning in multimodal large language models has two specific failure points. On the hardest queries, where models are most uncertain, training groups often produce unanimously wrong answers, which zeroes out the learning signal exactly when it's needed most. Separately, once a model becomes confident in a wrong answer, the math behind the gradient update shrinks toward zero, so confident mistakes get almost no correction. DEEPO patches both: it injects expert-written continuations on high-uncertainty questions to restore a usable training signal, and it reweights gradients so confident wrong answers still get pushed toward the right one.
This matters because hallucination has been the stubborn side effect of the reasoning-focused RL techniques, like GRPO, that AI labs increasingly rely on to make models reason step by step. Most existing fixes bolt on penalties or filters after training; DEEPO instead argues the problem is structural, baked into how gradients behave as a model's confidence grows. On the hardest long-video benchmark in the authors' test suite, the combined method beat the baseline by 4 points, with the two fixes proving additive rather than redundant.
It's one arXiv preprint with no code or model release mentioned, tested on the authors' own benchmark suite, so treat 'reduces hallucination' as promising math, not a shipped guarantee.