WavSLM, an autoregressive speech language model, has been released on arXiv. The authors quantize and distill WavLM’s self‑supervised embeddings into a single codebook, then train the model to predict the next audio chunk. The system works without any text labels or pre‑trained text models, yet it hits consistency benchmarks and produces intelligible speech.
The significance lies in returning to the vanilla next‑token paradigm that powers large text models, but now for audio. Prior speech models either stack separate text and acoustic pipelines, rely on hierarchical tokenizers, or inflate size to handle the dual information streams. By collapsing both layers into one token sequence, WavSLM trims parameter count and data requirements while still supporting streaming inference—a practical edge for real‑time applications.
In context, this mirrors the broader trend of simplifying multimodal models: Vision‑LLM hybrids are likewise converging on single‑stream tokenizers. Compared with earlier distillation efforts that still needed a text backbone, WavSLM’s complete bypass of text supervision could lower entry barriers for languages lacking large corpora. Whether the approach scales to larger datasets or more expressive vocoders remains to be seen, but it provides a proof point that speech generation can follow the same lean autoregressive playbook as text.
Bottom line: WavSLM shows that a stripped‑down, single‑stream architecture can keep pace with more complex speech models, hinting at a future where speech AI is cheaper to train and easier to deploy.