arXiv cs.AI· Xiaobo Wang, Zimu Wang, Siyu Ye, Jiaqi Li, Min Tang, Jun Bai, Yuling Shi, Ruobing Wang, Jingjing Zhang, Chaofan Wang, Kai Wu, Mengnan Qi, Kai Cai, Xiaodong Gu, Qi Liu, Zilong Zheng·· 12 小时前AI 评分38
智能体框架演进研究：基准分数提升未必带来可靠、高效、可迁移的改进
Beneath the Surface: What We Overlook When We Talk About Agent Evolution
AI 导读
研究评估了四种智能体框架演进方法在通用与编程任务上的表现，发现全部十六个演进后的智能体都获得了此前未覆盖的任务，同时丢失了此前已覆盖的任务。在一个通用任务案例中，可靠性提升 5.2 分却丢失了十项已覆盖任务，且多数可靠性增益同时推高部署成本。
正文 · 原文
Authors:Xiaobo Wang, Zimu Wang, Siyu Ye, Jiaqi Li, Min Tang, Jun Bai, Yuling Shi, Ruobing Wang, Jingjing Zhang, Chaofan Wang, Kai Wu, Mengnan Qi, Kai Cai, Xiaodong Gu, Qi Liu, Zilong Zheng
Abstract:Agent harness evolution enables recursive self-improvement (RSI) without updating model weights. We examine whether higher benchmark scores yield reliable, efficient, and transferable improvement. We evaluate four methods across harnesses, general and coding tasks. Repeated evaluations, controlled agent expansion, component interventions, and admission replays reveal recurring patterns. All sixteen evolved agents gain previously uncovered tasks while losing previously covered ones. In one general-task case, reliability rises by 5.2 points despite losing ten previously covered tasks. Most reliability gains also increase deployment cost. Specific changes usually outperform generic reminders, but their benefits can change when combined. The highest-scoring agent does not yield the largest mean improvement in any of our four pools, and stricter admission can discard useful alternatives. Transfer selectively preserves gains: coverage usually falls on the general-task target across both harness types and paradigms, but rises on the coding target with the outer-layer-only harness. These findings motivate preservation-aware evaluation, tested decision rules, and explicit applicability conditions. As a constructive follow-up, EvoAudit turns these diagnostics into revisions of select, analyze, mutate, and gate, achieving the best performance.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.07624 [cs.AI] |
| (or arXiv:2611.07624v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07624 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiaobo Wang [view email]
[v1]
Thu, 5 Nov 2026 08:06:37 UTC (2,334 KB)
来源：arXiv cs.AI · arxiv.org