Researchers have found a way to make audio-visual AI models dramatically cheaper to run without retraining them.
Omni-modal large language models can process audio and video simultaneously, but that joint processing generates token sequences long enough to make inference expensive. Most existing compression approaches lean on cues from a single modality — typically video — and assume audio and video share the same information density over time, an assumption that rarely holds. OmniFocus, a new training-free method from a team publishing on arXiv, sidesteps that by estimating the importance of tokens independently for each modality, guided by the actual query being asked. The result is a compression design that treats audio and video symmetrically rather than letting one dominate.
At 25% token retention — meaning 75% of tokens are discarded — OmniFocus still hit 59.40 accuracy on the DailyOmni benchmark using Qwen2.5-Omni-7B, while delivering up to a 1.38x prefill speedup over the full-token baseline. That matters because inference cost is the main bottleneck stopping multimodal AI from scaling cheaply in production; a method that works without retraining lowers the barrier to deployment significantly.
The approach outperforms existing baselines on several major benchmarks at the 25% retention threshold — though it was tested specifically on the Qwen2.5-Omni model family, so generalization to other omni-modal architectures remains an open question.