Randomly masking which words a language model can use during self-play training turns out to be enough to keep it from teaching itself nothing.
When two models are set up in a loop, one generating math problems and the other solving them, the generator quickly converges on a narrow set of prompts. It learns what satisfies the reward function and stops exploring. Researchers call this diversity collapse, and it quietly breaks the whole premise of autonomous curriculum learning. A team training Qwen3-4B and Qwen3-8B via a framework called R-Zero introduced vocabulary dropout: a hard, randomly-changing mask applied to the proposer model's output probabilities that prevents it from locking into fixed phrasing. Because the mask is non-stationary, the model cannot simply route around it. Diversity held up across lexical, semantic, and functional metrics throughout training.
The practical result at 8B parameters was a +4.4-point average gain on math benchmarks, with the largest improvements on competition-level problems where pattern-matching is least useful. Self-play is attractive precisely because training data costs nothing to label, but a lazy generator poisons the loop from the start. The authors frame the fix through an analogy to game rules: in chess, the rules themselves constrain the action space; here, an artificial constraint serves the same structural role.
The authors are careful to call vocabulary dropout one instantiation of a principle, not a solved problem. That framing is right: a randomly shifting mask is one collapse vector away from the next failure mode nobody has found yet.