Large language models are bad at knowing what they don't know-and that problem gets worse in non-English languages.
Researchers ran the first large-scale benchmark of uncertainty estimation (UE) methods across 22 languages, from well-resourced ones like Spanish and Chinese to low-resource languages with sparse training data. They tested nine different UE approaches-methods that help a model decide when to answer versus when to abstain-across multiple model sizes and architectures, using two human-curated question-and-answer datasets. The core finding: asking a model to reason in English, even when the question is written in a low-resource language, closes most of the reliability gap between rich and sparse-data languages.
That result reframes where the problem actually lives. Models aren't failing to understand low-resource languages-they're failing to generate reliable uncertainty signals in them. The bottleneck is on the output side, not the input side, which means a simple prompting change can substitute for years of additional training data. The study also found that the best UE method depends on model scale: smaller models do better with open-box probability-based techniques, while larger models are better served by closed-box self-verbalized uncertainty, where the model states its own confidence directly.
Uncertainty estimation rarely gets the spotlight that benchmark scores or context windows do, but it matters for any deployment where a wrong confident answer is worse than no answer at all-think medical triage tools, legal research assistants, or multilingual customer support. The multilingual gap in UE has been a known blind spot; this paper offers a practical workaround while the field waits for models trained on more balanced language data.