arXiv cs.AI· Chenqian Le, Jiayi Cheng, Qijia He, Runhao Li, Yinghao Li, Xupeng Chen·· 6 天前AI 评分29
多执行框架强化学习训练编码智能体：训练增益与框架依赖
Multi-Harness Reinforcement Learning for Coding Agents: Training Gains and Harness Dependence
AI 导读
研究用同一 SFT 检查点训练两个 Qwen3-8B RL 变体，仅奖励归一化方式不同（框架内 Within 与跨框架 Cross）。在 SWE-bench Verified 上，四种训练框架的平均提升分别为 +0.48 和 +0.77 个百分点；Cross 虽提高非零优势轨迹比例，但在未见框架上未稳定优于 Within，两者均未显著超过 SFT。
正文 · 原文
Abstract:Recent systems train coding agents with reinforcement learning (RL) on trajectories collected through multiple execution harnesses. However, gains under the training harnesses leave open whether the improvements transfer to unseen interfaces and how they depend on reward normalization. We present a controlled study that measures RL gains under fixed harnesses and compares the same models across harnesses. We train two Qwen3-8B RL variants from the same supervised fine-tuning (SFT) checkpoint using identical replay data. They differ only in whether rewards for each task are normalized within each harness (Within) or across harnesses (Cross). On SWE-bench Verified, we identify three main findings: (i) Multi-harness RL yields modest gains over the SFT baseline under both normalization rules. Across the four training harnesses, the estimated mean improvements are +0.48 and +0.77 percentage points for Within and Cross, respectively. (ii) Cross increases the fraction of trajectories with nonzero advantage but provides no consistent advantage over Within on an unseen harness across training seeds and evaluation budgets; neither rule significantly improves over SFT on this harness. (iii) Performance varies substantially more across harnesses than with RL training. After averaging over model variants, the highest and lowest source-harness solve rates differ by 7.13 percentage points, far exceeding the estimated training gains. Even after multi-harness RL, the observed performance differences across execution interfaces remain substantially larger than either the gains from training or the differences between normalization rules.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.04518 [cs.AI] |
| (or arXiv:2609.04518v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.04518 arXiv-issued DOI via DataCite |
Submission history
From: Chenqian Le [view email]
[v1]
Thu, 3 Sep 2026 22:17:02 UTC (472 KB)
[v2]
Fri, 9 Oct 2026 15:50:30 UTC (478 KB)
来源：arXiv cs.AI · arxiv.org