Alibaba's Qwen team has a new realtime voice model, and its biggest fix isn't accuracy - it's teaching the assistant to shut up when nobody's talking to it.
Qwen-Audio-3.1-Realtime, detailed in a paper updated on arXiv this week, is built from three pieces the team calls Think, Act, and Speak and Coordinate. Think blends supervised fine-tuning with multi-teacher distillation to convert the model's language skills into native audio understanding. Act trains it with reinforcement learning inside self-evolving executable environments, teaching it to call tools and act on feedback rather than just respond. Speak and Coordinate is the layer that decides whether, when, and how the assistant should actually speak or act. On a half-duplex speech-to-text benchmark, task success rose from 78.4 percent to 82.0 percent compared with the prior 3.0 version. On a full-duplex speech-to-speech benchmark, the rate at which the model wrongly responded to background speech fell from 73.0 percent to 13.0 percent.
That interruption fix matters more than the accuracy bump. Full-duplex voice assistants, ones that can listen and talk at the same time, are only usable if they can tell when they're actually being addressed instead of jumping in on a TV in the background or a stranger's conversation. The paper also describes a separate Voice Harness prototype that pairs the older 3.0 model with background coordination and memory to handle longer-running tasks, which hints the team wants this to become a standing agent, not just a chatty assistant.
Whether that agent stays polite and useful once it's juggling real chores instead of benchmark scripts is the harder question, and this paper mostly answers the easier one.