arXiv cs.CL· Yuanjian Xu, Jianing Hao, Tianze Sun, Yongyao Duan, Sam Chen, Jinyi Han, Wanbo Zhang, Yuxuan Sun, Guang Zhang, Zhong Li·· 2 天前AI 评分24
大语言模型的数据策略：训练全生命周期的统一视角
Data Strategies for Large Language Models: A Unified Perspective on the Training Lifecycle
AI 导读
这篇综述将 LLM 训练全生命周期的数据策略统一表述为在候选样本上分配训练预算的策略，并按决策粒度、模型依赖性和时间自适应性对方法进行分类。细粒度或状态依赖的策略控制力更强，但会带来额外的打分、搜索或 rollout 成本；跨模型规模的可迁移性和可比的端到端成本证据仍然有限。论文附带持续更新的论文列表与公开实现。
正文 · 原文
Abstract:Data strategy plays a central role in training large language models (LLMs). It governs the selection, weighting, and scheduling of training data, thereby determining the capabilities a model can acquire from a fixed data budget. We treat high-quality data as having high training-dependent utility, which depends on the target objective, the model, and its training state rather than on the text alone. Research in this area now spans pretraining and post-training, but related ideas use inconsistent terminology and evaluation protocols. This work reviews data strategies across the training lifecycle and formulates them as policies that allocate a training budget over candidate examples. We organise methods by their decision granularity, model dependence, and temporal adaptivity, then compare their computational requirements and empirical support. Fine-grained or state-dependent policies offer more control but incur additional scoring, search, or rollout costs. Evidence for transfer across model scales and comparable end-to-end cost remains limited. A continuously updated list of papers, along with their public implementations, is available at this https URL.
| Comments: | 21 pages, 5 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.21681 [cs.CL] |
| (or arXiv:2610.21681v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21681 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuanjian Xu [view email]
[v1]
Thu, 15 Oct 2026 11:08:14 UTC (7,371 KB)
来源：arXiv cs.CL · arxiv.org