A research team has built an AI framework that attempts to automate one of traditional Chinese medicine's oldest diagnostic tools: reading the tongue.
The system, called MMIR-TCM, chains together three components — a segmentation module that isolates the tongue from an image, a fine-tuned vision-language model (Qwen3-VL) that generates a structured diagnosis, and a retrieval-augmented generation layer that pulls in supporting clinical evidence before issuing a prescription recommendation. The team developed a new dataset, MedTCM, to train and validate the system, and also built a custom evaluation metric called TDEU because standard benchmarks couldn't capture what a correct TCM diagnosis actually looks like. In head-to-head tests, MMIR-TCM outperformed both GPT-4o and Gemini 2.5 Flash on those clinical accuracy measures.
The gap this targets is real: tongue inspection in TCM is notoriously subjective, and reproducibility across practitioners is poor. If a retrieval-backed multimodal model can produce consistent, evidence-grounded outputs, it could serve as a decision-support layer for clinicians rather than a replacement — a framing the field increasingly favors to sidestep regulatory friction.
The harder question is whether a custom metric that the authors designed themselves is the right yardstick. Outperforming frontier models on a benchmark you built is a result worth watching, not celebrating yet.