arXiv cs.CL· Zicheng Xu, Ruixuan Zhang, Yu-Neng Chuang, Xiuyi Lou, Hoang Anh Duy Le, Oren Gal, Alexander S. Szalay, Zhaozhuo Xu, Guanchu Wang, Vladimir Braverman·· 2 天前AI 评分22
自适应数据调度 ADS 如何提升 LLM 强化学习效果
Learning at the Right Pace: Adaptive Data Scheduling Improves LLM Reinforcement Learning
AI 导读
研究者提出自适应数据调度（ADS），一种双层数据调度框架，在语义聚类层面自适应调整簇间采样分布，在样本层面追踪接近策略能力边界的提示词。在三个 LLM 和七个推理基准上，ADS 相比 GRPO 平均准确率提升 5.5%，并能持续改进不同目标设计的 RL 方法，源代码已公开。
正文 · 原文
Abstract:Large Language Models (LLMs) achieve remarkable reasoning capabilities through reinforcement learning (RL) post-training. However, existing RL post-training commonly relies on uniform data sampling, which ignores the semantic structure of the training data and the evolving capability of the training policy. To address these limitations, we propose Adaptive Data Scheduling (ADS), a dual-level data scheduling framework that combines adaptive sampling across semantic clusters with policy-boundary sample selection. At the cluster level, ADS organizes samples according to semantic patterns and adapts the inter-cluster sampling distribution to prioritize clusters within the policy's current reach. At the sample level, ADS tracks prompts near the policy's capability boundary within each cluster to obtain informative relative advantages. Experimental results across three LLMs and seven reasoning benchmarks demonstrate that ADS improves average accuracy by 5.5% over Group Relative Policy Optimization (GRPO). Notably, ADS consistently improves RL methods with different objective designs, highlighting its potential as a general data scheduling strategy. The source code is available at: this https URL.
| Comments: | 15 pages, 4 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.22305 [cs.CL] |
| (or arXiv:2606.22305v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.22305 arXiv-issued DOI via DataCite |
Submission history
From: Zicheng Xu [view email]
[v1]
Sun, 21 Jun 2026 02:19:17 UTC (4,683 KB)
[v2]
Wed, 14 Oct 2026 23:58:09 UTC (4,697 KB)
来源：arXiv cs.CL · arxiv.org