arXiv cs.AI· Christoph Daxerer, Johann Glock, Christian Macho, Rudolf Ramler, Martin Pinzger, Josef Pichler·· 13 小时前AI 评分38
编码智能体还需要调试器吗？前沿模型在准确率天花板下的故障定位研究
Do Coding Agents Still Need a Debugger? Frontier-Model-Based Fault Localization at the Accuracy Ceiling
AI 导读
基于 Sol 6.1、Luna 6 和 Qwen 3.8 的编码智能体在 144 个 LeetCodeDataset 解法注入的 610 个单行故障上，无工具时即可定位原始程序 98.2%–99.9%、变换后程序 96.7%–99.9% 的故障。
正文 · 原文
Abstract:Background: To localize a fault, a coding agent can reason from source code alone or use debugger assistance by running the program and stepping through it in an interactive debugger. While earlier agents struggled with static execution tracing and debugger operation, frontier LLMs have improved at both. Objective: We evaluate how accurately coding agents localize faults without tools across original and transformed programs, how much debugger assistance improves accuracy and at what cost, and how agents use it when optional. Methods: We seed 610 single-line faults into 144 LeetCodeDataset solutions and rewrite each faulty program with five recipes of behavior-preserving transformations. Agents based on Sol 6.1, Luna 6, and Qwen 3.8 localize each fault with no, optional, or encouraged debugger assistance. Results: Without tools, the agents localized 98.2 to 99.9% of faults in original and 96.7 to 99.9% in transformed programs, losing accuracy mainly where recipes restructured loops and conditions. Encouraged assistance removed 63% and 61% of Luna 6's and Qwen 3.8's remaining transformed errors (+2.1 and +1.3 percentage points) at 14.7 to 27.0 times the token use, while Sol 6.1 stayed at 99.9%. With optional assistance, Luna 6 and Qwen 3.8 still gained +1.5 and +1.3 points, though the agents used the interactive debugger in only 0.2 to 6.2% of runs. Conclusions: On short, precisely specified programs, current coding agents localize nearly all faults from source code alone, and debugger assistance removes most errors that remain. Given the choice, agents executed the program far more often than they stepped through it, yet kept most of the benefit.
| Comments: | 12 pages, 3 figures, 5 tables |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36015 [cs.SE] |
| (or arXiv:2610.36015v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36015 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Johann Glock [view email]
[v1]
Mon, 26 Oct 2026 17:20:31 UTC (84 KB)
来源：arXiv cs.AI · arxiv.org