A frozen medical large language model can serve as a unified diagnostic layer across structured and unstructured hospital records without any retraining.
Researchers built a cohort of 13,645 admissions from the MIMIC-IV database, pulling from the 10 most frequent primary ICD-10 codes and grouping them into seven diagnostic categories. Instead of fine-tuning the model - a MedFound-Llama3-8B variant - they extracted hidden states from five transformer layers and trained lightweight linear probes on top. The combined probe, drawing on both discharge notes and serialized structured variables, hit 87.69% strict accuracy and 91.45% medical accuracy, beating single-modality probes and both an XGBoost baseline and a purpose-built clinical coding model called PLM-ICD. A 2-million-parameter adapter then transferred that capability to the older MIMIC-III dataset using only 5% of target labels.
The practical upside here is efficiency: clinical AI teams spend enormous resources fine-tuning models for every new data format or hospital system. If frozen embeddings can already separate diagnostic categories linearly - and the paper shows they get better at it in deeper layers - that cuts the adaptation cost substantially. The cross-dataset transfer result is the most useful finding, because real-world EHR systems are notoriously heterogeneous.
Automatic ICD coding has been a target for machine learning for over a decade with incremental gains; this paper's contribution is less about raw accuracy and more about showing that a shared representation space might replace a patchwork of task-specific models - a meaningful efficiency claim, if it holds outside controlled academic datasets.