arXiv cs.AI· Wenfeng Wang, Jianrui Shi, Mingang Guo, Jiayuan Ma, Yuqi Lu, Jiajun Sun, Guojiang Zhao, Jing Li·· 3 天前AI 评分37
Trace-to-Architecture：用执行轨迹诊断 LLM 智能体能力瓶颈并修订 harness
Trace-to-Architecture: Capability Bottleneck Diagnosis for Harness Revision
AI 导读
研究者提出 Trace-to-Architecture（T2A）框架，利用成功与失败任务的执行轨迹定位 LLM 智能体 harness 中需要修改的组件，并通过验证目标任务是否改善、已成功任务是否保持、编辑是否被触发来迭代修订。
正文 · 原文
Abstract:Improving an LLM agent requires knowing what to change when it fails. Similar failures can require different fixes, and resolving one problem can expose another. We introduce Trace-to-Architecture (T2A), a framework that uses execution trajectories to determine what to change in an agent's harness and when those changes apply. A development agent diagnoses failures using execution evidence from successful and failed tasks, revises the corresponding harness components, and then checks whether target tasks improve, selected successful tasks remain successful, and the edits are actually triggered. New execution evidence guides subsequent diagnosis and revision. Across ALFWorld, ScienceWorld, and AppWorld, T2A achieves the highest aggregate score for all three evaluated task agents, exceeding their respective strongest baselines by 6.2--30.3 points. We also integrate T2A's diagnosis and validation procedures into Meta-Harness, improving the resulting harnesses' mean ScienceWorld score by 21.6 points from the same starting harness.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16725 [cs.AI] |
| (or arXiv:2610.16725v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16725 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jianrui Shi [view email]
[v1]
Mon, 12 Oct 2026 12:56:13 UTC (1,884 KB)
来源：arXiv cs.AI · arxiv.org