arXiv cs.AI· Sebastian Bobadilla-Suarez, Bob Suh, Ryan Fortin·· 1 天前AI 评分32
审计脚手架而非检查点：智能体编程递归自我改进的平稳性二分法
Audit the Scaffold, Not the Checkpoint: A Stationarity Dichotomy for Recursive Self-Improvement in Agentic Coding
AI 导读
针对 AI 编程智能体的自我改进，研究提出审计重点应从模型权重转向其工具、测试与任务拆解构成的脚手架，因为脚手架可被重写而模型保持冻结。在公开编程基准上，每轮额外修订的收益趋近于零；来自同一模型族的 30 个智能体出现共同失败，多数投票在 55 个任务中有 23 个（42%）失败。研究强调证据为描述性，尚未运行所提出的验证实验。
正文 · 原文
Abstract:Rules for AI self-improvement often key on the model itself: has it been retrained, or are its weights frozen? We argue that, for AI coding agents, this check is incomplete, and we propose, but do not run, the experiment that would test it. An agent that keeps revising its own work runs into diminishing returns as long as the set of changes it can reach stays fixed, and can escape them only if that set grows. Our hypothesis is that the set can grow with the model untouched, because the software around the model, its scaffold of tools, tests and task breakdown, can be rewritten. If so, a frozen model guarantees a ceiling eventually but not steady behavior on the way there, and audits should track changes to the scaffold, not only to the model. The same criterion separates three things often lumped together: more search within a fixed setup, retraining the model during use (which can raise the ceiling), and rewriting the scaffold, which sits between them. Adding more agents does not lift the ceiling. Picking the best of several agents' answers reaches the best single agent's ceiling exactly, and only gets there faster. A majority vote could do better, but only if the agents fail on different tasks. In our tests, 30 agents from one model family failed together: some task defeated all of them, and a majority failed 23/55 (42%) of tasks. Our evidence is descriptive. On a public coding benchmark, the gain from each extra round of revision shrinks toward zero, though the unsolved tasks left to measure are also fewer and harder. Across 401 development sessions, the amount of code changed falls off from commit to commit, but programmers working before AI tools show the same pattern, so the data show that current practice levels off without showing why. We reach the criterion by treating repeated revision as a known machine-learning method, boosting, and note where real agents depart from it.
| Comments: | 10 pages main text, 47 pages total, 9 figures, 5 tables |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Software Engineering (cs.SE) |
| MSC classes: | 68T05, 68Q32, 68N30 |
| ACM classes: | I.2.6; I.2.2; D.2.5 |
| Cite as: | arXiv:2609.34924 [cs.LG] |
| (or arXiv:2609.34924v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.34924 arXiv-issued DOI via DataCite |
Submission history
From: Sebastian Bobadilla-Suarez [view email]
[v1]
Mon, 28 Sep 2026 11:25:52 UTC (178 KB)
[v2]
Thu, 12 Nov 2026 12:58:34 UTC (178 KB)
来源：arXiv cs.AI · arxiv.org