arXiv cs.AI· Yu Dong, Hong Li, Qiang Sun, Xu Zhou, Jinbiao Huo, Qixiu Cheng, Zhiyuan Liu·· 10 小时前AI 评分31
LM-SimBench：面向大语言模型训练仿真的统一框架、数据集与基准
LM-SimBench: A Unified Framework, Dataset, and Benchmark for Large Language Model Training Simulation
AI 导读
研究者推出 LM-SimBench，一个用于评估 LLM 训练仿真器的统一框架、性能剖析数据集与基准。该数据集在 Ascend NPU 集群上剖析了 10 个稠密与 MoE 大模型家族的 23 个模型设置，得到 348 个可行配置，联合变化五个并行维度和激活重计算，并提供六种粒度视图。匹配的 GPU-NPU 实验验证了该收集与评估协议的跨平台适用性，并展示了 NPU 引导的 GPU 配置选择。
正文 · 原文
Abstract:Evaluating candidate parallelization strategies through distributed training runs makes strategy selection prohibitively expensive at scale. Performance simulation offers a lower cost alternative, yet controlled comparisons among existing simulators remain difficult because they differ in the training stages they model and the workloads on which they are evaluated. We introduce LM-SimBench, a unified framework, profiling dataset, and benchmark for evaluating LLM training simulators. The framework decomposes simulation into functional stages from model compilation to schedule level aggregation, connecting execution representations and cost estimators to measurable evaluation targets. Following this framework, we profile 23 model settings from 10 dense and Mixture-of-Experts (MoE) LLM families on an Ascend Neural Processing Unit (NPU) cluster. The resulting dataset contains 348 feasible configurations that jointly vary five parallelism dimensions and activation recomputation, with six views spanning configuration, rank, operator, event, and kernel granularity. Evaluations of representative training simulators demonstrate how LM-SimBench supports comparisons of prediction accuracy, assessment of configuration selection, and diagnosis of prediction errors. Matched GPU-NPU experiments validate the cross-platform applicability of the collection and evaluation protocol and demonstrate NPU-guided GPU configuration selection.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.14285 [cs.AI] |
| (or arXiv:2611.14285v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14285 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yu Dong [view email]
[v1]
Tue, 10 Nov 2026 16:26:19 UTC (9,218 KB)
来源：arXiv cs.AI · arxiv.org