AI models that explain their reasoning step by step often aren't describing the reasoning that actually drove their answer.
Researchers adapted two existing tests for checking whether a language model's chain-of-thought explanation is honest, the Counterfactual Test and Correlational Counterfactual Test, so they work on vision language models, calling the new versions vCT and vCCT. The method uses image pairs that differ by exactly one object, then checks whether a model's stated reasoning tracks what actually changed its answer. Across eight open-source vision language models and two new benchmark datasets, Counter-SNLI-VE and Counter-A-OKVQA, the stated explanations frequently skipped the object that caused a big swing in the model's prediction, or cited an object that barely mattered. A control run that edited images without removing anything confirmed the shifts were tied to the real object removal, not to artifacts of the editing pipeline itself.
This adds to a growing pile of evidence that a model's chain-of-thought is closer to a plausible-sounding narration than a transcript of what happened inside it. The researchers also found that explanations generated after an answer track real evidence more closely than reasoning generated before the answer, which undercuts the common assumption that reasoning-first prompting makes a model's logic more inspectable.
For anyone using chain-of-thought output to judge whether a model's decision is trustworthy, this is a reminder that asking a model to show its work does not mean it will show the real work.