Bigger AI models can't undo bad audio compression, a new study finds.
Researchers built a framework called UniARC to systematically compare the two main ways audio gets fed into large audio language models: continuous features, which keep the raw signal mostly intact, and discrete tokens, which compress audio into a vocabulary the way text gets tokenized. They tested both approaches across speech, sound, and music tasks, and scaled the underlying language backbone from a 135-million-parameter SmolLM2 up to an 8-billion-parameter Llama-3. The team tracked how data volume, model capacity, and compute efficiency interact under each representation. The headline finding: tokenization schemes that preserve semantic meaning outperform ones that don't, and making the backbone bigger does not compensate for information lost when audio gets discretized, especially when training data is limited.
That's a direct challenge to the industry habit of treating scale as a universal fix. If a tokenizer throws away acoustic detail, no amount of extra parameters buys it back, which matters for teams deciding how to architect audio models rather than just how big to make them.
In other words, audio language models are still fighting the same discrete-versus-continuous battle text models settled years ago, and this paper argues you can't just scale your way out of picking the wrong side.