Researchers have built a system that adds a language model reasoning layer on top of traditional PET/CT tumor detection — and it outperforms image-only approaches on two test datasets.
The system, called RADIANT-PET, works in two stages. First, a deliberately permissive segmentation model flags every suspicious region in a scan — erring on the side of catching too much. Each candidate region is then converted into a structured text description covering uptake intensity, shape, and anatomical location. A large language model reads that description, along with the patient's radiology report if one is available, and decides whether the region is a real lesion or a physiologic false positive. To sharpen that judgment, the team fine-tuned the LLM using reinforcement learning, rewarding correct lesion classification and accurate anatomical site assignments. Results on the AutoPET benchmark and an Ohio State University test cohort showed consistent gains over image-only baselines, with the biggest improvements coming when radiology reports were included.
The core problem RADIANT-PET attacks is real: PET scanners measure metabolic activity, and normal organs like the brain and bladder light up the same way tumors do. Radiologists learn to discount those signals through anatomical knowledge and clinical context — something purely visual models struggle to replicate. Routing that contextual reasoning through a language model is a sensible structural fix, not a brute-force scaling play.
The catch is that the system's biggest gains depend on having a radiology report on hand — which means it shines in review workflows but may offer less lift for initial reads where no prior report exists.