A new paper argues that AI models fail to share knowledge across languages for a strangely mundane reason: they never see the same tokens twice.
Researchers pretrained 360M and 7B parameter language models and confirmed that cross-lingual knowledge transfer breaks down early, during pretraining itself, and does not improve with standard fixes applied afterward. To isolate the cause, they built a controlled experiment: two so-called languages that were actually the same language, sharing identical text and identical token segmentation, but assigned to separate, non-overlapping token vocabularies. Even with everything else held constant, the models still treated the two copies as unrelated, failing to apply facts learned in one to the other. The problem was not grammar or meaning, but simply that the tokens lived in different spaces.
That reframes a stubborn complaint about multilingual models: they behave less like one bilingual mind and more like several monolingual ones sharing a body. The fix the researchers propose, mapping languages onto a shared token space through simple word-for-word translation, recovered up to 12.6% of the efficiency a model gets from learning in its native language alone, 14 times better than the untranslated baseline. That is a tokenizer-level fix rather than a training trick, which matters for anyone deciding how to architect the next generation of multilingual models.
It is a useful reminder that some of what looks like an AI model's reasoning limits is really just leftover plumbing from how we chopped up the text in the first place.