Ask an AI to describe a song, and there is a decent chance it will confidently get the vocals wrong.
Researchers built a diagnostic framework called MuseDiag to test how nine audio-language models, four open-source and five closed-source, perceive music across five layers: sound events, timing, tonal attributes, style, and emotion. Using a contradiction-based verification method, they found vocal misperception was a universal weakness across every model tested, while tonal perception was the biggest factor separating the strongest systems from the weakest. Audio-Flamingo-3 held the top spot overall, but the models ranked below it reshuffled significantly depending on which testing paradigm was used. The team also tried two training-free fixes, ADD-M and TPA, which cut down hallucination in controlled probing tests.
This matches what casual users of AI music tools have probably already noticed: these systems sound sure of themselves even when they are guessing. The bigger point is that hallucination in audio models is not one bug to patch, but a stack of separate failures spanning perception, architecture, and how text gets generated, so no single fix covers all of it.
The two mitigation methods worked in lab-style probing but mostly fell apart once models were asked to generate free-form descriptions, the same gap that has dogged text-based hallucination fixes for years.