NemotronLabs has open-sourced a speech-to-speech AI model that listens, talks, and calls tools in one continuous stream, no turn-taking script required.
NemotronLabs built the model around a streaming speech encoder paired with a decoder-only language model that outputs two parallel streams at once: conversational text and structured function calls. A separate RNN-T branch transcribes the user's speech incrementally, while a streaming text-to-speech decoder handles the model's own voice, all inside one architecture rather than a chain of separate services. On Full-Duplex-Bench 1.0, it posts the lowest pause-handling takeover rate among the open-weight systems tested, meaning it rarely jumps in during an ordinary conversational pause, but it takes the floor 100 percent of the time once a user actually interrupts it, and its own post-interruption replies score 4.33 out of 5 for quality. On the newer Full-Duplex-Bench 1.5 and 3.0 suites, it resumes after a user's backchannel cue 93 percent of the time, scores 55.1 on VoiceBench's normalized average, and hits 82.5 percent tool-selection F1, though the paper flags argument accuracy and full tool execution as still weak.
That distinction matters because most voice assistants either fake full-duplex behavior with silence-detection heuristics or bolt tool-calling onto a transcribe-then-respond pipeline, adding latency at every step. NemotronLabs VoiceChat runs listening, reasoning, tool invocation, and speaking through one streaming model, which is the harder engineering problem and the one that actually matters for agentic voice apps that need to act mid-conversation. The tool-selection score is respectable, but the acknowledged gap in argument accuracy and execution is what decides whether this becomes a demo or a product.
Taking over completely after an interruption isn't necessarily a flaw (that is arguably what a good listener does), but it is also the kind of tidy number a benchmark report can make sound more impressive than it is until you check what it is actually measuring.