arXiv cs.AI· Mushir Akhtar, Hazem N. Nounou, Majdi Mansouri, Mohamed N. Nounou·· 2 天前AI 评分29
Muon 何时真正胜出？一项数据、时间与调参预算配对基准测试
When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark
AI 导读
在单张 A100 上，研究对比 Muon 与 AdamW 在 17.7M、33.6M 参数语言模型和 2.70M 参数视觉 Transformer 上的表现。
正文 · 原文
Abstract:Muon's gains in data efficiency motivate evaluating its performance under explicit data, time, and tuning budgets. On a single A100 GPU, we compare a reference single-device Muon implementation with AdamW across 17.7M- and 33.6M-parameter causal language models and 2.70M-parameter vision transformers. A common 23-candidate development budget selects configurations evaluated with five matched seeds, full validation trajectories, optimizer-step timing, and a frozen one-shot test. Muon reduces validation-loss area under the curve by 3.25--22.44\% at equal data and lowers test loss in all 20 primary pairs. In this configuration, Muon improves time-normalized loss area} by 19.68--22.31\% on vision, while AdamW is better by 3.38--6.96\% on language. The full Muon optimizer call, including auxiliary AdamW updates, consumes 56.1--69.6\% of training time, compared with 6.1--9.1\% for the AdamW baseline. Muon also has higher expected regret under restricted access to these candidate pools. These results characterize small models with reference kernels; their wall-clock ordering does not establish a ranking for larger models or optimized implementations.
| Comments: | Accepted at NeurIPS 2026 Workshop (Optimization for Machine Learning) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.21555 [cs.LG] |
| (or arXiv:2610.21555v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21555 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mushir Akhtar [view email]
[v1]
Thu, 15 Oct 2026 09:15:49 UTC (302 KB)
来源：arXiv cs.AI · arxiv.org