arXiv cs.AI· Heewon Park, Somin Im, Minhae Kwon·· 10 天前AI 评分33
SAG：面向长时程决策的成本感知 LLM 智能体选择性批评框架
Selective Critique for Cost-Aware LLM Agents in Long-Horizon Decision Making
AI 导读
研究者提出 SAG（Self-improving Agent with Gated critique），一种成本感知框架，将批评调用建模为长时程交互中的逐步决策问题，通过全局熵和局部 top-2 margin 等动作级歧义信号实现免训练门控，近似批评的信息价值（VoI）。
正文 · 原文
Abstract:Improving the reliability of large language model (LLM) agents in long-horizon decision-making remains a key challenge. When deployed as autonomous agents interacting with complex environments, early mistakes can propagate through trajectories and cause cascading failures. Recent approaches improve reliability by incorporating external critique or deliberation, but invoking these mechanisms at every step substantially increases token consumption and latency, limiting practical deployment. We propose SAG (Self-improving Agent with Gated critique), a cost-aware framework that formulates critique invocation as a step-wise decision problem during long-horizon interaction. SAG introduces a lightweight, training-free gating mechanism that estimates the utility of critique using action-level ambiguity signals--global entropy and local top-2 margin--computed over admissible actions. From a decision-theoretic perspective, this mechanism approximates the Value of Information (VoI) of critique, enabling the agent to selectively allocate expensive feedback only when its expected benefit justifies the cost. SAG further incorporates online bootstrapped self-improvement, allowing the actor to internalize critic-assisted behaviors and progressively reduce reliance on critique. Across three long-horizon interactive benchmarks and multiple backbone models, SAG substantially improves the performance-cost trade-off compared with both no-critique and always-on critique agents. On ALFWorld, SAG increases task success from 24.6% to 78.4% while maintaining a token budget comparable to ReAct, yielding a $3.1\times$ improvement in normalized token efficiency. Moreover, a 7B actor with a lightweight 3B critic achieves performance comparable to a 14B actor without critique, showing that selective critique can recover most of the reliability benefits of deliberation while dramatically reducing inference cost.
| Comments: | Accepted to NeurIPS 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.07335 [cs.LG] |
| (or arXiv:2610.07335v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.07335 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Heewon Park [view email]
[v1]
Mon, 5 Oct 2026 20:09:48 UTC (3,040 KB)
来源：arXiv cs.AI · arxiv.org