Vision-language models have a habit of sounding confident while ignoring the image in front of them.
A new paper out of the arXiv preprint server proposes a fix called Faithful Warm-Start (FWS): a preparatory training stage that runs before reinforcement learning kicks in. The researchers built a dataset called FaithfulQA by pulling samples from six visual question-answering benchmarks, selecting only image-question pairs that require genuine visual reasoning — not ones a model could answer from language patterns alone. A second model then acts as a judge, filtering the dataset further for causal consistency between image and answer. Only after this warm-start phase does the standard reinforcement learning step begin, using sparse answer-level rewards.
The core problem the paper addresses is well-documented: reinforcement learning makes models more fluent, but fluency is not accuracy. A model trained purely on whether its final answer is right can learn to generate plausible-sounding reasoning traces that barely reference the visual input. FWS attempts to route around that failure mode by instilling visually grounded habits before the RL optimization has a chance to exploit language shortcuts. The authors report improvements in answer accuracy, more stable training runs, and fewer reasoning steps that float free of visual evidence.
The approach is essentially a curation and filtering play dressed up as a methodology — which is not a criticism, since bad training data is the root cause of most of these grounding failures. Whether FaithfulQA generalizes beyond the six benchmarks it was drawn from is the question worth watching.