跳到正文
arXiv cs.AI· Jingxuan Xu, Gang Wu, Yanan Wu, Yutao Mou, Songwei Yu, Tianzhuang He, Zhengshuo Gong, Zhao Liu, Zihang Xu, Wenqiang Zhu, Xinping Lei, Weihao Li, Ziqi Miao, Yuhui Bai, Zhongqiu Wang, Yan Wu, Ariel Deng·· 4 小时前AI 评分38

FLARE:基于生成式奖励模型的长程编程智能体全生命周期稠密监督范式

FLARE: A Full-Lifecycle Dense Supervision Paradigm for Long-Horizon Coding Agents via Generative Reward Model

AI 导读

FLARE 提出一种由轻量生成式奖励模型(GRM)驱动的稠密监督范式,用于长程软件工程智能体。其 RADAR 框架通过因果链回溯提取无事后信息的监督信号,蒸馏出的 GRM 在推理时拦截高风险步骤做断点重执行,FLARE(N=1)在 token 消耗降低 5 倍的情况下优于 Global Rollout(N=5)。用于训练后,SFT 相对提升 19.13%,RL 稳定提升 9.19%。

正文 · 原文

Authors:Jingxuan Xu, Gang Wu, Yanan Wu, Yutao Mou, Songwei Yu, Tianzhuang He, Zhengshuo Gong, Zhao Liu, Zihang Xu, Wenqiang Zhu, Xinping Lei, Weihao Li, Ziqi Miao, Yuhui Bai, Zhongqiu Wang, Yan Wu, Ariel Deng

View PDF HTML (experimental)

Abstract:While test-time scaling enhances Large Language Model (LLM) agents in long-horizon software engineering (SWE), sparse binary rewards (Pass/Fail) create a severe credit assignment crisis and waste failed exploratory trajectories. Current trajectory optimization and scaling methods are costly and structurally limited, relying on heuristic state reuse without causal diagnosis or delayed scalar scoring without actionable online guidance. We propose FLARE (Full-Lifecycle Alignment and Reward Engine), a novel dense supervision paradigm driven by a lightweight Generative Reward Model (GRM). First, RADAR, an offline causal-aware diagnostic framework, extracts high-fidelity, hindsight-free supervision through causal-chain backtracking to distill a GRM providing real-time, step-level risk feedback. Second, FLARE uses this GRM to continuously optimize the agent across its entire lifecycle. During inference, FLARE acts as an Active Scaffold, autonomously intercepting high-risk generation steps for localized breakpoint re-execution, drastically reducing compute overhead. During post-training, the GRM's structured signals serve as process-supervised reranking scores for Supervised Fine-Tuning (SFT) and step-level dense rewards for Reinforcement Learning (RL), mitigating policy collapse in sparse environments. Extensive evaluations show that FLARE establishes a new Pareto frontier across the agent lifecycle: FLARE (N=1) outperforms Global Rollout (N=5) with a 5x reduction in token consumption. Extending FLARE to training overcomes the sparse reward problem in long-horizon interactive tasks, delivering relative performance gains of 19.13% in SFT through process-aware data curation and a consistent 9.19% improvement in RL.
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2609.23808 [cs.CL]
  (or arXiv:2609.23808v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2609.23808

arXiv-issued DOI via DataCite

Submission history

From: Miao Ziqi [view email]
[v1] Sun, 20 Sep 2026 18:54:16 UTC (2,530 KB)
[v2] Tue, 17 Nov 2026 09:59:52 UTC (2,507 KB)

来源:arXiv cs.AI · arxiv.org