A refinement to the Muon optimizer trims training overhead without touching hyperparameters or model architecture.
Muon and similar orthogonality-based optimizers have gained traction in large-scale training benchmarks, but they carry a notable cost: a gradient orthogonalization step that relies on Newton-Schulz iterations, which typically require dozens of matrix multiplications to converge. Researchers behind Turbo-Muon introduce a pre-conditioning procedure that improves how those iterations are initialized, reducing the initial polar error enough to eliminate one of the five Newton-Schulz iterations normally run in practice. The result is a measurable drop in overhead with negligible cost added by the pre-conditioning itself. The code is live on GitHub and available in optax and Hugging Face kernels, as the paper describes.
The 3% training-time reduction may look modest on a slide deck, but at the scale where frontier labs run thousands of GPU-hours per run, that margin compounds fast. More practically, it drops in as a replacement with no retuning — the kind of tradeoff that actually gets adopted in production pipelines rather than shelved after a benchmark. The authors also offer a theoretical account of why the update resists feature collapse, which gives practitioners a reason to trust it beyond the numbers.
Orthogonality-based optimizers are still a relatively recent entrant competing against entrenched adaptive methods like AdamW; a frictionless speed gain is exactly the kind of evidence that moves adoption — though whether these benchmark improvements hold across every architecture and data regime remains an open question.