AI/ ai · machine-learning · llm · research

Filtering by Surprise Slows AI Model Collapse

Researchers find that fine-tuning LLMs on high-perplexity documents can curb the diversity loss that occurs when models train on their own outputs.

A new paper proposes a perplexity-based filter to slow the self-reinforcing decay that hits AI models trained on AI-generated text.

As synthetic content crowds out human writing on the web, large language models face a growing risk of training on their own prior outputs — a feedback loop the researchers call AI autophagy. The paper, which ran simulations across multiple datasets and LLM families, finds that this loop causes models to pile probability mass onto a shrinking set of tokens, narrowing the range of things they will say. Alongside that diversity loss, the researchers also measured a decline in commonsense inference accuracy, suggesting the damage runs deeper than stylistic repetition. The key diagnostic insight is that fine-tuning on low-perplexity documents — text the model finds unsurprising — accelerates collapse, while prioritizing high-perplexity, "surprising" documents during fine-tuning consistently slowed it.

The practical appeal of this approach is that it sidesteps a problem that haunts most existing mitigation strategies: needing to tell human-written content apart from AI-generated content, a task that is getting harder as detectors lag behind generators. A filter that works on what the model finds surprising is model-centric and needs no content labels. That matters because the volume of unlabeled synthetic text in training pipelines is only going up.

The approach still depends on fine-tuning, not pretraining from scratch, so how well it holds at the scale of frontier model training runs remains an open question — and one the labs with the biggest pipelines have every incentive to study quietly rather than publish.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →