Researchers built a benchmark that checks AI-generated audio against instructions one requirement at a time, instead of asking another AI "does this sound right?"
The new system, called AnyAudio-Judge, breaks complex audio prompts into a variable set of independent, verifiable yes-or-no rubric items, then scores generated audio against each one. The team built the AnyAudio-Judge Bench, a bilingual test set of 7,920 samples spanning speech, sound, music, and mixed audio, with hard negatives designed to trip up sloppy graders. They also trained a dedicated evaluator model on 105,000 examples that include step-by-step reasoning, using a mix of supervised fine-tuning and Group Relative Policy Optimization so its judgments track the rubric checklist rather than a vague overall impression. In testing, the model outperformed existing zero-shot evaluation baselines at catching alignment mismatches.
Most audio-generation evaluation today outsources judgment to a general-purpose large language model, which returns a single opaque score and no way to tell which part of a prompt got ignored. A rubric-based, interpretable score matters because it can double as a training signal, feeding cleaner reinforcement-learning rewards back into the audio generators themselves, not just grading them after the fact.
It is the same lesson image and video generation learned already: one big vague score does not tell you what actually broke, so eventually somebody builds a checklist.