A new arxiv paper argues that the reasoning steps AI models print out don't always match what's actually happening inside them.
The paper, titled "Chain-of-Thought Reasoning in the Wild Is Not Always Faithful," was posted to arxiv under the identifier 2503.08679. Chain-of-thought prompting asks a model to write out intermediate reasoning steps before giving a final answer, a technique widely credited with improving accuracy on math and logic tasks. This paper's central claim is that, outside of controlled benchmarks, those written steps are not a reliable record of the computation the model actually used to reach its answer. It surfaced on Hacker News, where it drew a modest 10 points and four comments.
If a model's stated reasoning is decorative rather than diagnostic, that's a problem for anyone treating chain-of-thought output as a debugging tool or a trust signal. Vendors building reasoning models have leaned on visible step-by-step output partly to reassure users that a system's logic can be inspected. A finding that this window can be unfaithful chips away at one of the more comforting stories the industry tells about interpretability.
Chain-of-thought was sold as a peek inside the black box. This paper is a reminder that a model narrating its reasoning and a model actually using that reasoning are not necessarily the same thing.