arXiv cs.CL· Kai Mei, Zhiyuan Hu, Yutong Dai, Juntao Tan, Yifan Zhang, Dingjie Song, Dimitris N. Metaxas, Silvio Savarese, Ran Xu, Zeyuan Chen·· 9 天前AI 评分34
Opera：面向长时程编程智能体的语言化评论框架
Opera: A Verbal Critic Framework for Long-horizon Coding Agents
AI 导读
Opera 是一个语言化评论框架，把每次纠正当作持续跟踪的笔记，直到被诊断的问题真正解决，通过周期性及事件触发决定何时审查、用类型化算子诊断问题、在交付前对照可见证据审核反馈，并追踪智能体后续动作以区分表面服从与真正解决。
正文 · 原文
Abstract:Long-horizon coding agents need timely corrections, yet feedback can be ineffective or even harmful when it misjudges ongoing work or fails to address the underlying problem. Existing critics focus on evaluating trajectories and generating feedback, but rarely track what happens after feedback is delivered. We present Opera, a verbal critic framework that treats each correction as a persistent note, followed until the diagnosed problem is resolved. Opera decides when to review through periodic and event-driven triggers, diagnoses issues with typed operators, audits feedback against visible evidence before delivery, and tracks the agent's subsequent actions to distinguish mere compliance from actual resolution. As a test-time critic, Opera improves the resolve rate of non-critic agents by up to 12.4, 15.0, and 8.9 percentage points on Terminal-Bench 2.1, a SWE-Bench Pro subset, and DeepSWE v1.1, respectively, across four policy models, and achieves the highest mean resolve rate among competitive critic baselines on all three benchmarks, and also improves policy models when the policy critiques itself. Beyond inference, Opera-guided rollouts provide approximately on-policy training data: fine-tuning Qwen3.5-9B on them improves its resolve rate on held-out SWE-Bench Pro repositories by 10.2 percentage points without a critic at inference time, matching fine-tuning on rollouts from a stronger model, while preserving its performance when switching harness, i.e., from Openhands to Terminus-2, which the latter substantially degrades. Our code is available at: this https URL.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.33987 [cs.CL] |
| (or arXiv:2609.33987v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.33987 arXiv-issued DOI via DataCite |
Submission history
From: Kai Mei [view email]
[v1]
Sun, 27 Sep 2026 22:42:57 UTC (1,346 KB)
[v2]
Thu, 8 Oct 2026 11:11:53 UTC (1,346 KB)
来源：arXiv cs.CL · arxiv.org