Large language models are getting a lesson in intellectual humility.
Researchers have published a paper introducing reinforcement learning with metacognitive feedback (RLMF), a training approach designed to fix one of the more stubborn problems in AI: models that state wrong answers with complete confidence. The method works in two stages. First, it calibrates how accurately a model scores its own performance — essentially grading its own work. Then it translates those internal confidence scores into plain-language hedges like "I'm not sure" or "this might be wrong." A companion technique, metacognitive data selection, uses the same self-assessment signal to pick better training examples, beating standard active learning approaches.
The reason this matters is that hallucination isn't just a accuracy problem — it's a trust problem. A model that's wrong but sounds certain is harder to catch than one that flags its own doubt. RLMF outperformed standard reinforcement learning by up to 63% on faithful calibration benchmarks, without degrading accuracy on the underlying tasks. That's a meaningful gap, and it suggests self-assessment quality is a viable training signal in its own right.
Frontier models from every major lab currently struggle to align expressed confidence with actual knowledge boundaries. Whether this approach scales to production systems — or survives contact with the adversarial prompts that real users send — is the question this paper leaves open.