LLMs don't just absorb human prejudice — they can manufacture their own.
Researchers tested whether large language models would develop biases against demographic groups that had no real-world differences built into the experiment. They did. Drawing on a framework from psychology, the study found that models spontaneously formed skewed impressions of artificial groups and then acted on them, producing task allocations more stratified than those made by human participants. Newer and larger models made the problem worse, not better. The mechanism mirrors a well-studied human cognitive failure: too little exploration early on lets a handful of observations harden into assumptions about entire groups.
The dominant approach to LLM bias has been to scrub training data and fine-tune away known prejudices. This research argues that approach has a blind spot: it treats bias as a fixed artifact to remove, not as something a model can generate fresh from its own decision-making patterns. As LLMs move from text generators into systems that allocate resources, screen applicants, or route support tickets, that distinction stops being academic.
The researchers tested a range of fixes — tweaking inputs, restructuring problems, steering outputs — and found most had limited effect. The one intervention that worked reliably was explicitly incentivizing the model to explore before settling into a pattern. That's a narrow solution to a wide problem, and it signals that current bias-mitigation checklists may need a serious rethink before these systems are handed any real authority.