AI audio models can describe a soundscape but often can't tell you exactly when the dog barks.
Researchers have introduced Auto-AEG, a data construction pipeline designed to teach large audio-language models to localize sound events in time — given a plain-language description like "glass breaking," the model predicts the precise start and end timestamps. The bottleneck until now has been data: manually annotating when sounds begin and end in real recordings is slow and expensive, and most existing datasets only cover a fixed list of sound categories. Auto-AEG works around both problems by pairing programmatically synthesized audio clips — which carry exact ground-truth timestamps for free — with pseudo-labels generated by multiple models on real-world audio, then using reinforcement learning to sharpen the result.
This matters because the gap between "understanding" audio and actually using that understanding in practice is largely a localization problem. A model that can say "there is a siren somewhere in this clip" is less useful than one that can say "the siren runs from 4.2 to 7.8 seconds" — the latter is what you need for video editing, accessibility tools, surveillance pipelines, or any application that has to act on a specific moment. The team also releases AEGBench, a difficulty-stratified benchmark meant to give the field a consistent yardstick.
Results on both AEGBench and the established DESED detection benchmark show gains over baselines, though the paper is a preprint and independent replication is still ahead. The broader pattern here is familiar: when labeled data is scarce, synthesize it and let the model sort out what's real — a playbook that has worked in text and vision, now being applied to the timeline of sound.