A new paper argues that AI's hallucination problem and its safety-filter problem are actually the same problem, and neither gets fixed until benchmarks stop punishing a model for saying it doesn't know.
The paper, posted to arXiv this week, lines up three findings that looked unrelated. One 2026 study found that training models with reinforcement learning on reasoning tasks breaks their internal sense of when a tool call can be trusted. A second 2026 study found that when generation is constrained for safety, large models quietly rewrite flagged spans of text while small models just truncate them. A third result, a 2024 theorem, proved that any consistent reasoning system without a built-in way to say "I don't know" will keep hallucinating infinitely often on certain classes of problems, no matter how capable it gets. The new paper's argument is that all three trace back to one missing piece: a calibrated way for a model to decline to answer.
That missing piece is not primarily a training problem, it is a scoring problem. Dominant AI benchmarks assign zero reward to declining to answer, so there is no gradient in the leaderboard that would ever select for a model that knows when to stay quiet. Honesty post-training has narrowed the gap in deployed models, the paper says, but it cannot close a gap that a theorem says is structural.
The fix on offer is deliberately unglamorous: score answers three ways instead of one, report how often models decline, break out results by task difficulty, and require calibration metrics. Boring is probably the point, since the flashier fixes to date have mostly produced models that guess more confidently rather than guess less.