Researchers found that replacing standard token embeddings with raw images of Chinese text consistently outperforms the conventional approach across multiple model architectures.
The study swapped out index-based token embeddings entirely, feeding a single rasterized image of each character sequence into a vision encoder built from a shared ResNet and a shallow Vision Transformer. To keep the comparison clean, both the image-based model and the token-based baseline shared the same decoder backbone, training objective, optimizer, and data. The image model hit 0.429 accuracy versus 0.355 for the baseline — a 21% relative gain — and got there in roughly half the training epochs. The edge showed up fast: within the first five epochs, covering less than 21% of total training data.
This matters because tokenization is one of those foundational assumptions in NLP that almost nobody questions. If images of characters can outperform learned token embeddings for a major writing system, it reopens the question of whether the field optimized around an English-shaped constraint from the start. The finding that a corrupted image model still matches a clean token-based baseline adds practical weight to the claim.
The advantage does not carry over to English or other alphabetic scripts — the researchers attribute the gap to the visual density and radical structure of Chinese characters specifically — so don't expect image-based tokenization to replace subword models anytime soon. But for Chinese-language AI, this is a nudge worth taking seriously.