A research technique that selects audio perturbations adaptively pushes large audio-language models closer to what they actually hear — not what they expect to hear.
Large audio-language models hallucinate in a specific way: they let language priors override acoustic evidence, confidently describing sounds that were never there. Contrastive decoding is a training-free workaround that feeds the model a degraded "negative" audio branch alongside the real one, nudging it toward the genuine signal. The catch is that existing methods use blunt tools — random noise, simple masking — when structured distortions would work better. This paper evaluates a library of targeted audio perturbations across temporal, spectral, frequency, and amplitude domains and routes each example to the transformation that hurts the model most usefully. Reversing the audio array, for instance, wrecks temporal coherence in a way that raises accuracy on temporal-order tasks from 74.7% to 81.4%.
Two findings stand out. First, a simple binary yes/no constraint in the prompt reduces the model's tendency to falsely confirm audio features that were never present — a cheap editorial fix that improves baseline behavior before any perturbation is applied. Second, a lightweight selector trained on model hidden states dynamically picks the best negative branch per example, delivering an additional 4.3 percentage-point gain on the existence task — telling whether a sound is present at all.
The no-retraining requirement matters here: enterprises already running audio models at scale can slot this in without spinning up a new training run. Whether the gains hold outside benchmark conditions is the question no benchmark can answer.