Researchers have released ClinOCR-Bench, the first publicly available benchmark designed to evaluate how well AI models read scanned clinical documents.
The dataset contains 384 scanned images split across six subsets — Normal, Handwriting, Poor Quality, Rotation, Tables, and Mix-artifacts — covering the kinds of degraded, rotated, and handwritten pages that routinely defeat standard text-extraction tools. It strips out protected health information, uses a template-aware train/test split to prevent data leakage, and includes baseline results from both open-weight and proprietary vision language models. The whole thing is on GitHub.
Most clinical OCR research has run on private institutional data, which means published results are nearly impossible to reproduce or compare. A shared, realistic benchmark changes that: labs can now measure their models against the same messy fax-era documents, track progress over time, and identify which artifact types — rotated pages, handwritten fields, low-resolution scans — still break modern systems. That matters because poor OCR is a quiet tax on electronic health records, causing transcription errors and manual rework downstream.
Vision language models have been closing in on traditional OCR pipelines, but without a common test set, that claim has been mostly vibes. ClinOCR-Bench won't settle the debate, but it at least puts the contestants in the same room.