Researchers just showed that the AI models being tested for self-driving cars often can't tell when their own vision is failing.
A new study tested five vision-language models - Qwen3.5-9B, Gemma4-E4B, LLaVA-OneVision-7B, DriveFusion/DriveFusionQA-4B, and NVIDIA's Alpamayo-1.5-10B - on four driving-related question-answering datasets. The team fed each model degraded visual inputs meant to mimic real-world sensor noise and bad weather, across single-frame, multi-view, multi-frame, and monocular camera setups. Accuracy dropped unevenly across models, datasets, and conditions, and in several cases the models' confidence scores stayed high even as their answers got worse. The researchers also tried Visual Evidence Augmentation, an inference-time fix, and found it helped in some model-dataset combinations but not others.
In a system where a wrong guess can mean a wrong turn into traffic, a confidence score that lies is arguably worse than no confidence score at all. This isn't really a story about which model is smartest - it's about whether any of these systems can flag their own uncertainty, which is the property safety engineers actually need before handing over control.
Until that gets fixed, bolting a bigger vision-language model onto a car's sensor stack is a feature demo, not a safety upgrade.