A new technique lets AI vision models stop wasting their internal reasoning power on redundant lookups.
Researchers describe MoLE, short for Mixture of Latent Experts, in a new paper on vision-language models - AI systems that look at an image and reason about it using hidden numerical placeholders called latent tokens, instead of writing out their thinking in words. The problem: in older setups, every latent token drew from the same pool of visual information through an identical pathway, so bolting on more tokens just produced near-duplicate work. MoLE fixes that by walling off each token's access to visual evidence during the extraction step, then using separate "summary" tokens to merge those different views back together. The whole thing trains in two stages - first forcing all visual information through this specialized pathway, then reopening direct access - without anyone having to hand-assign what each token should focus on.
The payoff shows up on the scoreboard. Across five visual reasoning benchmarks, MoLE averaged a score of 78.6, beating a standard fine-tuned version of the same model by 4.9 points and the best rival latent-reasoning method by 3.6 points at the same token budget. The researchers also found the specialized tokens attended to more varied parts of an image and looked less alike internally than before, evidence the specialization is doing real work rather than just relabeling the same computation.
It is a useful data point against the assumption that AI models get smarter mainly by handing them more scratch space to think in. Here, more tokens only helped once they were forced to stop repeating each other. Worth remembering: this is a preprint result on benchmark scores, not a shipped product, and a single average of 78.6 across five tests papers over a messier underlying spread.