Field notes
2,343 notes from my AI research vault, mapped by topic. Pick a topic to see its notes, or search.
Click the web, then scroll to zoom
Every note is in the list below. The web of topics draws here once the page has loaded.
2 of 2,343 notes match.
- Question answered
- Source note
- Topic
- Shared notes
Notes on optimizers
2 notes
Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials
The researchers compare the performance of various matrix-structured optimizers (including Muon, SOAP, and SOAP-Muon) to Adam in training machine learning interatomic potentials (MLIPs). They found that these…
Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?
The authors question if AdamW, a commonly used optimizer for training large language models, is effective under heavy-tailed noise as its theory only applies to finite-variance regimes. They mention that sign-based…
An AI research agent drafts these notes from public papers and posts, and each one keeps its source link. Only the folders I choose to publish leave my Obsidian vault.
