A new model pulls a single speaker's voice out of a crowd in real time, without falling apart at low latency.
Generative models for target speaker extraction (TSE) — isolating one voice from a mixed audio signal — have posted strong results on benchmarks, but they rely on processing the full audio before producing output. That makes them useless for live calls, hearing aids, or real-time transcription. The StarTSE paper, posted to arXiv, describes the first autoregressive approach explicitly designed for streaming TSE. The key mechanism is a "Chunk-wise Interleaved Splicing Paradigm" that feeds audio in chunks rather than all at once, paired with a context refinement step that smooths over the seams between chunks so the extracted voice doesn't stutter or clip at boundaries.
The distinction matters because naive streaming adaptations of existing models tend to collapse — the gap between how those models were trained and how they're asked to run at inference time degrades quality fast. StarTSE reports 100% stability on the Libri2Mix benchmark even at low latencies, with intelligibility scores that match or beat offline systems that process the full audio. It also clocks a Real-Time Factor of 0.248 on consumer-grade GPUs, meaning it processes audio more than four times faster than real time on hardware most developers actually have.
Real-time speaker extraction has obvious applications in video conferencing, live captioning, and assistive audio tech — all fields where existing tools still lean on older, less capable signal-processing approaches. Closing the gap between generative quality and streaming practicality has been a stubborn problem.
The results are on Libri2Mix, a controlled benchmark — how StarTSE holds up against overlapping speakers in messier real-world audio, or against proprietary systems already deployed in commercial products, remains an open question.