A team of researchers found a way to make a general-purpose AI model write specialist colonoscopy reports without retraining it at all.
The approach, called a context-fusion framework, pairs a frozen vision-language model with two lightweight additions: a self-supervised polyp encoder that pulls up similar past image-report pairs as evidence, and a small set of learned tokens that carry standing instructions about how to describe polyps. Neither touches the underlying model's weights. Tested on 2,056 expert-annotated endoscopic images, the framework beat both plain general-purpose VLMs and models fine-tuned specifically for the task. It did so while adding trainable parameters equal to just 0.006% of the frozen model's size, and when its top retrieved case matched the correct diagnosis category, it fixed 70.5% of the errors made by a full weight-adaptation baseline.
That's a meaningful data point for anyone worried about the cost of specializing AI for medicine. Fine-tuning a model on medical images is expensive, slow to update, and risks degrading the model's general reasoning. Bolting on retrieval and a handful of learned tokens instead sidesteps both problems, and the retrieval step gives clinicians a paper trail showing which past cases informed a given report.
Still, this is one condition on one public dataset of 2,056 images, not a hospital deployment. The retrieval-plus-prompting trick is basically the RAG playbook applied to medicine, and it will need to prove itself on messier, real-world endoscopy footage before anyone hands it a scope.