Researchers have found a way to shrink vision-language models more aggressively without the usual accuracy collapse, by teaching the compression process to tell important image data from filler.
The method, called SalQ-VLM, targets post-training quantization, the standard technique for shrinking a trained model's memory footprint by representing its internals with fewer bits. The researchers identified two problems existing quantization methods ignore: vision-language models tend to process far more image tokens than necessary, and there is a persistent gap between how text and image data are represented internally. SalQ-VLM addresses both by calculating which tokens most affect the model's output and weighting the compression process to protect those while discarding precision on the rest. It runs entirely during calibration, meaning it adds no extra computation when the model is actually being used, and works with existing inference software. Tested across multiple benchmarks and model backbones, the technique held up notably well at extreme compression levels, improving accuracy by 16.45% on the MME-RealWorld benchmark at 2-bit quantization compared to prior methods.
This matters because vision-language models are getting pushed toward phones, cameras, and other devices that cannot run billion-parameter models at full precision. Most quantization research has focused on text-only language models, leaving image-handling models under-optimized despite carrying extra baggage in the form of redundant visual tokens. A method that specifically prunes that redundancy, rather than compressing everything uniformly, addresses a gap the field has mostly talked around rather than solved.
The 2-bit result is the headline number, but 2-bit quantization is also where most models fall apart, so a 16% jump there says more about how bad the baseline was than how good SalQ-VLM is at everyday compression levels.