A new research paper describes a way to train a single language model that can run at 1, 2, 3, or 4 billion parameters, picked on the fly based on the device it's deployed to.
Researchers built a unified framework that merges two separate efficiency tricks: elastic architectures, which nest smaller sub-networks inside a larger one, and sparse mixture-of-experts routing, which activates only the parameters relevant to a given input. The model conditions on both the input context and a target efficiency setting, then routes computation through the elastically-nested sub-networks to hit that target. In testing, the resulting model matched the latency of same-sized dense models while beating those dense models by 2-5% on knowledge-intensive benchmarks, and it matched the accuracy of separately-trained static versions at each size.
Today, serving a range of device types usually means training and shipping several separate models, like a 1B, 3B, and 8B version of the same base model, each with its own full set of weights to store and maintain. Because this approach shares parameters across all capacity points in one model, it cuts on-device disk space and lets a server pick a size based on whatever DRAM and compute happen to be available, which matters as more inference shifts onto phones and laptops with tight memory budgets.
It's one arXiv paper with the authors' own benchmarks, not an independent audit or a shipped product, so the accuracy gains are worth watching rather than banking on.