A new benchmark finds that frontier AI models can discuss medicine from a scan, but keeping patients safe is another matter.
Researchers introduced IMCBench, a benchmark that pairs real, publicly available clinical images with synthetic patient profiles to simulate multi-turn patient-clinician exchanges. Eight models across four families (Claude, GPT, Nova, and Llama) were scored on safety, accuracy, and calibrated uncertainty on a 1-5 scale, using an LLM-as-jury system calibrated against expert clinician annotations. The paper identifies the top performer as a model its authors designate Claude Opus 4.6 (3.61 overall), followed by models they label Claude Sonnet 4.6 (3.30) and GPT-5.2 (3.29); those are the researchers' own designations and have not been independently verified against official product lineups. No model led on every dimension.
The headline finding is not the leaderboard order; it is the safety drop. Scores fell by 0.27 points for both malignant and rare conditions, and removing either visual input or electronic health record context made models measurably less safe (average drops of 0.18 and 0.23 respectively). That gap between accuracy and safe guidance is precisely what earlier single-turn, text-only benchmarks could not surface.
Most medical AI evaluations test whether a model can answer a question correctly. IMCBench tests whether it would give advice a clinician would stand behind. Current frontier models have not cleared that bar.