arXiv cs.AI· Christoph Treude, Sebastian Baltes·· 15 小时前AI 评分34
编码智能体的 harness 如何自我进化？一项关于回顾式适配的实证研究
How Do Coding-Agent Harnesses Self-Evolve? An Empirical Study of Retrospective Adaptation
AI 导读
研究以 Claude Code 在 30 个 SWE-bench Verified 问题上运行，用名为 \retro 的技能让智能体回顾已完成会话并提出 harness 修改，共收集 125 条提案、实现 24 条，并在同一仓库的后续问题上评估 12 项改动。
正文 · 原文
Abstract:Coding agents can adapt their harnesses, the software, configuration, and instructions used to run them, based on experience from earlier tasks, but whether these adaptations improve their behavior on later tasks remains unclear. We study \retro, a skill that asks an agent to review a completed session and propose harness changes, and ask what changes it proposes and whether implemented changes reduce repeated work on later tasks. We ran Claude Code on 30 SWE-bench Verified issues, qualitatively coded the 125 proposals from \retro, implemented 24, and evaluated 12 harness changes on later issues from the same repositories in fresh sessions with and without each change. Most proposals referred to an event in the session, but 18 of 43 lint-related proposals recommended a practice regardless of what had happened. Among the seven evaluations in which sessions without the change encountered the targeted problem, two changes eliminated the targeted action, one reduced its occurrence, four showed no reduction, and some changes added work on unrelated issues. Harness self-evolution requires more than generating and implementing changes: it needs ways to establish whether a change helps on later tasks and to revise or remove changes that do not.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.34938 [cs.SE] |
| (or arXiv:2610.34938v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34938 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Christoph Treude [view email]
[v1]
Mon, 26 Oct 2026 07:06:31 UTC (11 KB)
来源：arXiv cs.AI · arxiv.org