A new pruning method lets vision-language models drop most of their image tokens before the language model ever processes them, without retraining anything.
The technique, called VPRune, is a training-free pruning framework that runs before an image reaches the language-model half of a vision-language system. It targets three specific failure modes that make aggressive token pruning risky: bias from letting the accompanying text steer which visual tokens survive, information loss from tokens getting discarded outright, and positional distortion once the remaining tokens get compacted into a shorter sequence. VPRune counters these with three steps - selecting for visual diversity without leaning on the text prompt, recycling similar tokens instead of deleting them, and preserving original position information during restoration. Tested on the FastVLM-1.5B model across multiple benchmarks, it held onto more accuracy than blunter pruning approaches, especially at aggressive compression levels, and it cut end-to-end latency on edge devices too.
Vision-language models are expensive partly because images turn into hundreds or thousands of tokens, far more than a typical text prompt, so pruning has become one of the main levers for squeezing these models onto phones or embedded hardware. Most existing methods either trust the text prompt to decide what visual information matters, which introduces bias, or drop tokens outright, which loses information. VPRune's choice to recycle rather than discard, plus its fix for the position problem, is a more careful answer to a tradeoff other methods have mostly ignored.
The results so far are on one 1.5B-parameter model, so it is worth watching whether the gains hold up on larger or differently built vision-language systems before anyone treats this as a general fix.