Most AI "conformity" research may be measuring the wrong thing.
Researchers tested six open-weight large language models across seven question-answering and reasoning datasets, introducing a condition where the same wrong answer was repeated in the prompt but stripped of any attributed speaker. That alone caused models to abandon a correct answer 66.5% of the time — compared with just 10.3% when the model was simply asked the question again without any pressure. The effect held even when the wrong answer was paraphrased and when answer choices were hidden in open-ended settings. Expert-panel framing pushed the flip rate higher; minimal person labels made little difference.
The finding exposes a flaw in how AI conformity has been measured. Standard benchmarks bundle two signals together — a speaker and a repeated wrong answer — and vary them as one, making it impossible to isolate social influence from simple text repetition. The implication is that what researchers have been calling peer pressure may mostly be a sensitivity to repeated text, not anything resembling human social conformity.
When models did flip, they flipped confidently, and standard recalibration did not recover the original correct answer. Speaker attribution still matters, the authors argue — it just needs to be measured as an increment above this baseline "speaker-free floor" rather than treated as the whole story. Labs benchmarking model robustness against sycophancy may need to go back and re-examine their methodology.