跳到正文
arXiv cs.AI· Nicolas Chapados, Jean-Fran\c{c}ois Gagn\'e, Nouha Dziri·· 10 小时前AI 评分38

Metrics-in-the-Loop:复杂度反馈重塑智能体编写的代码

Metrics-in-the-Loop: Complexity Feedback Reshapes Agent-Written Code

AI 导读

研究提出 metrics-in-the-loop 推理时干预方法,在编码智能体每轮改代码后注入自动计算的代码指标反馈,提示圈复杂度超限的函数或被测试跳过的改动行。在三个长周期编码基准和四个强模型上的实验显示,复杂度反馈显著降低代码侵蚀(复杂度集中度指标),且不影响留出集正确性。

正文 · 原文

View PDF HTML (experimental)

Abstract:Coding agents now write whole codebases in long autonomous runs, guided mainly by whether their tests pass. Yet passing tests alone does not reveal accumulating complexity or gaps in test coverage, which then persist as agents extend their own code. We introduce metrics-in-the-loop, an inference-time intervention that gives agents in-flight feedback: cheap, automatically computed code metrics injected into the agent's context during the trajectory. After each code-changing agent turn, we insert short feedback messages naming changed functions whose cyclomatic complexity exceeds a limit (complexity feedback) or changed lines that are skipped by tests (coverage feedback), giving the agent the opportunity to refactor code or strengthen tests before continuing. We run extensive experiments and ablations comparing several interventions across three long-horizon coding benchmarks and four strong models. We find that complexity feedback significantly reduces erosion, a measure of complexity concentration of code, without affecting held-out correctness of the code. Broadly, our experiments investigate a practical form of agent "re-railing", where timely, actionable measurements redirect agents before weaknesses become the foundation they keep building on. As agents scale to codebases with millions of lines, such interventions become critical to sustaining effective development and long-term code maintainability.
Comments: Agent Evaluation Science Symposium, Fall 2026, New York City
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.23817 [cs.AI]
  (or arXiv:2611.23817v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.23817

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Nicolas Chapados [view email]
[v1] Wed, 18 Nov 2026 04:19:48 UTC (468 KB)

来源:arXiv cs.AI · arxiv.org