LLMs used in clinical settings can sense how well a medical claim is supported — they just won't tell you.
Researchers tested 22 open-weight language models against 45,134 clinical claims drawn from six public sources, harmonizing over 20,000 of them into a four-level evidence grading system. A simple linear estimator could recover the evidence grade from each model's internal activations with a median AUROC of 71.8 — meaning the signal was there. When the same models were asked to state that grade outright, their accuracy dropped to near chance, landing 25 to 27 percentage points below what the estimator pulled from their own representations.
The gap matters because clinical AI is already being used to summarize research and surface treatment guidance. If a model cannot reliably communicate how strongly evidence supports a claim, clinicians reading its output have no way to weigh it correctly — and the model is not going to volunteer that caveat. The study also found the signal was largely lexical, meaning it was tied to specific word patterns rather than genuine reasoning, and it did not transfer across topics or grading frameworks.
The findings cut against a broader assumption in medical AI: that scale fixes everything. Larger models did not decode evidence strength better, and reasoning-specialized models actually performed worst. The field keeps shipping clinical summarization tools; it has not yet shipped clinical summarization tools that know what they do not know.