An arXiv paper published today describes Audex, a 30-billion-parameter model that handles audio and text in a single unified architecture without the usual tradeoff of audio capability coming at the expense of language quality.
Audex is built on top of Nemotron-Cascade-2-30B-A3B, a mixture-of-experts text model. The team encodes audio inputs and projects them into the same embedding space as text tokens, then treats both uniformly during generation. That single-decoder design is notably straightforward — no separate audio tower, no complex routing between modalities. Training drew on 157.4 billion audio tokens and 320.5 billion text tokens, with multi-stage supervised training followed by reinforcement learning and on-policy distillation. The paper claims state-of-the-art results across speech recognition, translation, text-to-speech, and audio generation. Model checkpoints are publicly released.
The harder problem here is not adding audio capability — it is doing so without degrading the text reasoning that makes a large model useful in the first place. Most multimodal extensions quietly regress on benchmarks the base model already cleared; this paper claims "marginal or no regression" on reasoning, alignment, and agentic tasks, which, if it holds up on independent evaluation, is the actual result worth watching. Open weights mean the community can stress-test that claim quickly.
The affiliation behind the "Nemotron-Labs" name in the model string is not confirmed in the paper itself — whether this is an Nvidia team, an independent lab, or a project alias is unclear from the source material alone.