Training a large language model mostly in English may be quietly capping how good it gets at English.
A new study ran 220 controlled supervised fine-tuning experiments on models up to 8 billion parameters, using parallel translated data mixtures across math reasoning and API-calling tasks. The result: English-only post-training was consistently the weaker choice. Adding even one additional language to the training mix improved both English performance and the model's ability to generalize across languages. Piling on more languages helped further, with the biggest gains showing up for low-resource languages, while high-resource languages held steady instead of losing ground.
Most production post-training pipelines still treat non-English data as a bolt-on, if they include it at all, on the assumption that English-first keeps English sharp. This study argues that assumption has it backwards - multilingual data acts more like a subsidy for English performance than a tax on it. Diverse language mixtures also produced strong zero-shot transfer to languages the models never saw during training, meaning labs may not need to source fine-tuning data for every target language individually.
The catch: gains stayed limited for languages that are both low-resource and typologically distant from the training mix, so this is not a fix for the world's most underserved languages - just evidence that the industry's translate-to-English-first habit was a worse bet than most assumed.