Researchers found that swapping out the default optimizer in physics simulation models can meaningfully improve both training speed and accuracy.
A new paper tests matrix-structured optimizers — SOAP, Muon, and a hybrid SOAP-Muon — against Adam, the near-universal default for training machine learning interatomic potentials (MLIPs). MLIPs are models that predict how atoms interact, standing in for expensive quantum chemistry calculations in materials science and drug discovery. The researchers ran comparisons on two established MLIP architectures, NequIP and Allegro, and found that SOAP and SOAP-Muon consistently beat Adam on both convergence speed and final accuracy. Muon alone offered only partial gains.
The finding matters because optimizer choice has been a blind spot in this field. Nearly all MLIP research energy goes into new architectures and larger datasets; the paper argues that the training algorithm itself is an underexplored lever — and a cheap one, since switching optimizers requires no new data or model redesign. The gains were especially large when training with limited force supervision, which is relevant because force labels are expensive to generate.
Adam has dominated deep learning optimization for over a decade, and dethroning it in any domain is news. That said, MLIP benchmarks are notoriously narrow, and whether these results transfer to the largest foundation-scale simulation models remains an open question.