Ask a chatbot to answer a survey question as ten different personas, and you may get the same answer ten times.
A new arXiv paper tests "silicon sampling," the practice of using language models as stand-ins for human survey respondents, where each model call is supposed to act like an independent draw from a persona's likely response. Across three model families with different post-training pipelines, the researchers found every instruction-tuned model gave the identical answer to the same question on more than half the items in a public-opinion benchmark. The model's internal probabilities weren't spread across plausible answers - they piled up almost entirely on one option. Base models, before instruction tuning, showed far less of this collapse.
That matters because "synthetic respondents" are being pitched as a cheaper substitute for real polling panels, and this finding says the substitute isn't actually sampling anything - it's repeating itself. The paper also finds a workaround: the same models that can't sample from a distribution can describe it accurately when asked directly, cutting error against real survey data by more than half compared to averaging repeated persona calls. For cases that still need answers from individual personas, the authors propose a method called Prompt-Perturbed Argyle that trims error by another 21 percent at no added cost.
Alignment training built to make chatbots consistent and predictable turns out to be very good at its job - too good, if what you wanted from it was a crowd.