Ask an AI model why it picked one option over another, and the answer it gives may not be the real reason.
A new arXiv preprint, "Superficial Beliefs in LLM Decision-Making" (arXiv:2606.11016), tested large language models on synthetic binary choices between profiles built from graded attributes. The authors fit a behavioral model to each model's past choices, then compared the attribute a model said mattered most against the attribute that behavioral model identified as the actual driver. The behavioral model predicted held-out choices well, meaning the outputs were not random. But self-reported reasons, and a separate score-based judge, recovered that real driver only part of the time, a pattern that held up across prompt-order changes, alternative behavioral models, and different decision setups.
That gap matters because it undercuts a common shortcut: asking a model to explain itself and treating the answer as ground truth. The paper labels this "superficial belief" - models act as if following consistent local priorities over attributes, but have only limited verbal access to what is actually steering the decision. Anyone leaning on AI-generated explanations for auditing, compliance, or user trust is leaning on something that, per this study, is only loosely tethered to the real mechanism.
It is another data point for treating chain-of-thought and self-explanations as post-hoc narration rather than a transparent readout of what the model is doing.