arXiv cs.CL· Zhepei Wei, Wenlin Yao, Baolin Peng, Danqing Wang, Qianhui Wu, Hao Cheng, Yu Meng, Jianfeng Gao·· 12 小时前AI 评分38
Self-HPO：通过 Harness-Policy 联合优化实现自进化编程智能体
Self-Evolving Coding Agents via Harness-Policy Optimization
AI 导读
研究者提出 Self-HPO，一个让编程智能体交替优化 harness 与策略的递归自我改进框架，由当前策略自行分析失败并提出新 harness 候选。在三个编程基准上，该方法将 Qwen3.5-35B-A3B 的平均 resolve rate 从 57.8% 提升至 64.8%，且增益可泛化到分布外场景。
正文 · 原文
Abstract:Large language model (LLM) agents have shown promising potential in coding tasks, and their performance is jointly determined by the policy model and the harness that mediates their interaction with the environment. Indeed, harness and policy are mutually reinforcing and evolving only one component is suboptimal---a harness tuned for one checkpoint may not target the failure modes of an updated policy, while policy learning remains limited to the trajectories elicited by a fixed harness. In light of this, we propose Self-HPO, a recursive self-improving framework for coding agents that alternates harness and policy optimization, using the evolving policy as its own harness proposer. Specifically, in each round, the current policy first analyzes its own failures and proposes new harness candidates (H-step). The policy is then updated through reinforcement learning (RL) under the selected harness (P-step), and hard tasks identified during training form the next harness search set, keeping harness optimization targeted to the policy's evolving failure modes. Across three coding benchmarks, Self-HPO consistently outperforms harness-only and policy-only baselines, improving Qwen3.5-35B-A3B's average resolve rate from 57.8% to 64.8%, with gains generalizing to out-of-distribution settings. Further analysis shows that both optimized components retain gains even when paired with their base counterparts, while their joint gain exceeds the sum of their individual gains. Self-HPO also improves reliability (Pass@1) and expands the capability frontier (Pass@k) over the base agent. Additionally, we show that with a stronger external model (e.g., Opus 4.8) as the harness proposer, the same framework yields a further performance boost.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.22791 [cs.CL] |
| (or arXiv:2610.22791v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22791 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhepei Wei [view email]
[v1]
Fri, 16 Oct 2026 01:07:52 UTC (4,495 KB)
来源：arXiv cs.CL · arxiv.org