Large language models can diagnose complex clinical cases at a 60-70% accuracy rate — but a new paper suggests they may be guessing well rather than thinking clearly.
Researchers developed what they call clinical reasoning graphs: structured representations of how LLMs work through diagnostic problems, built from free-text traces using a defined ontology of five node types and seven edge types. They ran the pipeline across 750 traces from five different models, tested on 50 cases from the New England Journal of Medicine's Clinicopathological Conference series, under three prompt conditions. The core question was whether models apply consistent reasoning schemas to clinically similar cases. The answer was no. Within-cluster and between-cluster graph similarity were nearly equal, and no comparison survived multiple-testing correction.
The finding matters because it exposes a gap between what accuracy benchmarks measure and what clinical reliability actually requires. A model that reaches the right answer through different reasoning paths each time is not a model you can audit, trust, or safely deploy in a diagnostic workflow. Notably, correct and incorrect model pairs showed nearly identical graph similarity scores — 0.488 versus 0.484 — meaning the structure of the reasoning process offers almost no signal about whether the answer is right.
Structured reflection prompts did boost explicit discriminating-feature analysis by 33%, but did not improve cross-case consistency. That is a useful distinction: prompting can make a model look more methodical without making it more reliable. As AI developers push harder into clinical settings, this is the kind of process-level evaluation the field has largely skipped in favor of leaderboard scores.