arXiv cs.AI· Jiajun Hu, Chun Yong Chong·· 6 天前AI 评分41
SWE-Critix：面向编程智能体轨迹评估的通用验证器后训练方法
SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories
AI 导读
SWE-Critix 是一套后训练流程，用于训练能判断编程智能体执行轨迹并生成自然语言分析解释的通用验证器。该流程结合 SFT 与 RL，基于 CodeForge-Preview 轨迹训练了一系列 30B 模型，在留出测试集上将分类准确率从基础 Qwen3-30B-A3B 推理模型的 57% 提升至 SFT 后的 66%、RL 后的 78%。模型、数据集与训练代码均已开源。
正文 · 原文
Abstract:Coding agents are increasingly deployed for end-to-end software engineering tasks, where a readily available oracle evaluator that defines the success criteria is often absent. This motivates verifications for coding agents that can reliably judge and explain their behavior. However, verification is non-trivial as coding agents are increasingly capable of solving long-horizon software engineering tasks. Their execution trajectories can be extremely long, containing hundreds of iterative interactions with code repositories and external tools. Existing verifiers either rely on general-purpose LLMs without specialized training or train dedicated critics that predominantly focus on producing scalar rewards, highlighting the relatively limited exploration of training-based generalist verification. We introduce SWE-Critix, a post-training pipeline for training generalist verifiers of coding-agent trajectories. Given a task and its execution trajectory, the learned verifier predicts the completion label and generates a natural-language analysis that explains its judgment. Our pipeline combines supervised fine-tuning (SFT) with reinforcement learning (RL) and trains a series of 30B models on trajectories from CodeForge-Preview. On a held-out test set, SWE-Critix improves classification accuracy from 57% for the base Qwen3-30B-A3B reasoning model to 66% after SFT and 78% after RL. We release our models, datasets, and training code to facilitate further research.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.13356 [cs.AI] |
| (or arXiv:2610.13356v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13356 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiajun Hu [view email]
[v1]
Fri, 9 Oct 2026 10:30:21 UTC (273 KB)
来源：arXiv cs.AI · arxiv.org