A team of researchers just proved that audio-visual segmentation models have been carrying a lot of dead weight.
The new model, called EASE (Encoder-only Audio-Visual Segmentation), strips out the decoder stage that most Transformer-based segmentation systems inherited from image models. Detailed in a paper posted to arXiv (arXiv:2609.29121), EASE identifies, segments, and classifies sound-emitting objects in video frames using only an encoder. It runs at up to 365 frames per second - three times faster than the previous state-of-the-art at similar accuracy - and trains in under 11 GPU-hours. The team says EASE also hits state-of-the-art segmentation scores across multiple backbones and input resolutions.
Audio-visual segmentation underpins things like real-time captioning, surveillance analytics, and AR overlays that need to know which object on screen is making which sound. A model that trains in 11 GPU-hours instead of days lowers the barrier for smaller labs to iterate on this work, and a 3x speed jump matters for anything that has to run live instead of getting processed after the fact.
Code and model weights are posted at https://ease-avs.notion.site, which is more than most papers claiming to be faster and better bother to share.