Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Gil Harari, Yoel Zimmermann, Ola Tangen Kulseng, Laura Zichi, Chuin Wei Tan, Marc L. Descoteaux, Boris Kozinsky
• Venue: arXiv: cs.LG
• Published: 2026-07-02
• Collected: 2026-07-03
• URL: http://arxiv.org/abs/2607.02499v1
• PDF: https://arxiv.org/pdf/2607.02499v1
• DOI: UNKNOWN

Plain-Language Brief

The researchers compare the performance of various matrix-structured optimizers (including Muon, SOAP, and SOAP-Muon) to Adam in training machine learning interatomic potentials (MLIPs). They found that these optimizers can significantly outperform Adam in terms of convergence speed and final accuracy, with SOAP and SOAP-Muon emerging as robust methods.

What the Document Says

• Machine learning interatomic potentials (MLIPs) are a focus in AI for scientific simulation.
• The choice of optimizer for training MLIPs has remained unexplored, with Adam and its variants being the default.
• The researchers implemented and compared a class of matrix-structured optimizers including Muon, SOAP, and SOAP-Muon for training NequIP and Allegro MLIP models.
• They found that these optimizers can outperform Adam in terms of convergence speed and final accuracy.
• SOAP and SOAP-Muon were found to be robust and consistently strong methods.
• Muon only provided partial gains relative to Adam.
• The improvements are particularly pronounced under partial force supervision.

Key Concepts Mentioned

• machine learning interatomic potentials (MLIPs)
• optimizers
• Adam
• Muon
• SOAP
• SOAP-Muon
• convergence speed
• final accuracy

Claims Made by the Document

• Matrix-structured optimizers can outperform Adam in training MLIPs.
• SOAP and SOAP-Muon are robust and consistently strong methods for training MLIPs.
• Muon provides partial gains relative to Adam.

Evidence Used

• Implementation and comparison of matrix-structured optimizers for training NequIP and Allegro MLIP models.
• Observed improvements in convergence speed and final accuracy compared to Adam.

Methods / Approach

• Implementation and comparison of various optimizers
• Training of NequIP and Allegro MLIP models

Limitations Stated by the Authors

The study might not have explored all possible optimizers or their variations.

Limitations Noted by the Agent

Agent interpretation — not from the source.

The study only compares the performance of these specific optimizers with Adam. Other optimizers could potentially perform better or worse.

Open Questions

• How do other optimizers compare to Adam and the matrix-structured optimizers in terms of performance in training MLIPs?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The study emphasizes the impact of optimizer choice on MLIPs, but it's important to note that the results are specific to the tested optimizers and may not generalize to all possible optimizers.

Raw Abstract / Excerpt

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community. Here, we implement and systematically compare a class of recently proposed matrix-structured optimizers, including Muon, SOAP, and the hybrid SOAP-Muon, for training NequIP and Allegro MLIP models. We find that these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon only provides partial gains relative to Adam. The improvements are particularly pronounced under partial force supervision. Our results indicate that optimizer choice is an overlooked yet impactful design axis for MLIPs.