arXiv cs.CL· Taiheng Pan·· 12 小时前AI 评分42
从 RL 论文到框架代码：七大开源训练框架的收录与评审机制研究
From RL Papers to Framework Code: Inclusion and Review in Seven Open-Source Training Frameworks
AI 导读
研究追踪七个开源 RL 后训练框架的提交历史，并将其与 2025 年 1 月至 2026 年 6 月间 6,939 篇经筛选的 arXiv 论文关联，发现 1,598 篇提出通用 RL 训练方法的论文中仅 37 篇被至少一个框架收录。
正文 · 原文
Abstract:Practitioners use reinforcement learning methods for language models through training frameworks, so a method that a framework includes is one configuration change away for its users. We measure how often this happens. We trace seven open-source frameworks for RL post-training through their commit histories and link them to 6,939 screened arXiv papers from January 2025 to June 2026. Of 1,598 papers that propose a general RL training method, 37 reached at least one framework. Across five quarterly cohorts of papers, the number included within 180 days stayed between 5 and 7 while the cohorts grew from 72 to 386 papers. Compared with papers from the same quarters, included papers more often trained policies of 30B parameters or more, at 41 against 10 percent, and more often changed a coefficient, clip, mask or weight in the existing loss, at 29 against 9 percent. Both gaps hold without papers from the companies that maintain the frameworks, while code release was equally common in both groups. The frameworks' issues and pull requests show that few methods are turned away in public. More than nine in ten of the general methods no framework included have no thread there that cites or names them, and many proposals that do appear stay open without a reply. Two further records, the frameworks' issues and the technical reports of 13 large open-weight models, show that users rarely report training failures in public and that the reports state 8 of the 13 widely included methods. We release the data as a tracker that links training failures to framework implementations and lists each framework's contribution rules and earlier proposals.
| Comments: | 29 pages (13 pages main text), 5 figures, 12 tables. Data and code at this https URL and this https URL |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL); Software Engineering (cs.SE) |
| Cite as: | arXiv:2610.28027 [cs.LG] |
| (or arXiv:2610.28027v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28027 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Taiheng Pan [view email]
[v1]
Tue, 20 Oct 2026 11:16:01 UTC (122 KB)
来源：arXiv cs.CL · arxiv.org