AI/ ai · speech-ai · llm · research

Speech AI Model Writes Text and Talks at the Same Time

A new speech AI approach called Listen-Write-Speak has the model write full text responses while it talks, letting it code and reason mid-conversation.

A new speech model can talk out loud and write full text at the same time, instead of squeezing every answer through spoken words alone.

Researchers describe a system called Listen-Write-Speak (LWS), which uses a single language model that listens to audio, writes visible text, and speaks a spoken reply in parallel, all under one shared attention context. The trick is a "Token Schema" that organizes the three channels without requiring any changes to the model's architecture, paired with training data built from synthesized per-second annotations. On benchmark tests, LWS scored 4.72 on VoiceBench AlpacaEval and hit 92.6% consistency between what it wrote and what it said out loud. It also beat its own stripped-down versions on URO-Bench and performed well on Full-Duplex-Bench, a test of real-time back-and-forth conversation.

The catch with voice assistants has always been that speech can't easily carry code, tables, or multi-step reasoning - you can't dictate a working Python function over the phone. By making text a first-class output alongside audio, LWS lets a spoken assistant show its work: a block of code, a structured comparison, a chain of reasoning, while still replying out loud without a lag. That's a real gap for anyone using voice interfaces for anything more technical than a to-do list.

Whether this becomes the standard architecture for voice assistants or stays a research paper is the open question - plenty of promising full-duplex tricks never leave the benchmark stage.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →