Show a language model a stray number before you ask it a question, and it will quietly bend its answer toward that number - even when the number is nonsense.
A new benchmark called AnchorBench tests this systematically. Researchers ran fourteen models, ten open-weight and four frontier API systems, through a battery of controlled prompts that plant an anchor value through different channels, then measured how much each model's answer shifted. The design separates anchors that look plausible from ones that are obviously irrelevant, and tracks how the pull weakens as the planted number gets farther from the correct answer. The pattern held across the board: stronger anchor pathways produced bigger shifts, plausible-looking anchors moved answers more than nonsense ones, and the effect faded fastest in externally sourced and retrieval-augmented settings.
The uncomfortable finding is that accuracy scores hide this weakness. Models that answered over 95% of anchor-free questions correctly, including frontier API models, still bent toward planted numbers once one appeared. That matters anywhere an LLM estimates a price, a risk score, or an outcome after reading documents that happen to contain unrelated figures, since retrieval-augmented pipelines feed exactly that kind of stray context by default.
It is the same trick that has worked on human judgment since Kahneman and Tversky's wheel-of-fortune experiments in the 1970s, and apparently nobody patched it out of the machines either.