Multiple photos of the same skin lesion should make AI diagnosis easier. New research shows current tools can't tell which photo to trust.
A study spanning six dermatology datasets and nine frozen AI backbones tested "reliable-input selection" - having a system pick the best of several clinical photos of the same case before classifying it. When an oracle that could see the correct answers chose the best image, weighted F1 accuracy jumped by about 20 percentage points on average. But real-world selectors can't see labels. They have to judge reliability blind, using only what the model exposes at inference: embeddings, embedding norms, and confidence scores. Researchers benchmarked four such training-data-free methods, including confidence scores and neighborhood consensus among a case's images, and even tried adding a small labeled reference set.
This matters because teledermatology already runs on exactly this scenario: patients or referring clinicians submit several photos of a lesion, taken at different angles, distances, and lighting. Everyone assumes picking the clearest shot helps. This paper shows that intuition is right in theory but currently unbuildable in practice.
The best method, a fusion of model confidence and Mahalanobis distance, still left most of that 20-point gap on the table. Call it a well-documented dead end for now - useful mainly as a benchmark for whoever solves it next.