arXiv cs.CL· Hanwen Li, Jinhao Duan, Guanhua Zhu, Junchi Lu, Bo Shen, Chenxi Yuan, Kaidi Xu·· 10 天前AI 评分27
从不确定性到行动：学习引导 LLM 智能体
From Uncertainty to Action: Learning to Steer LLM Agents
AI 导读
研究提出 VoS（Value of Steering），一种轨迹级监控器，可从约 82,000 条反事实延续构成的逐步结果表（SOT）中学习每步引导价值，并用带伤害预算的触发器决定是否引导。VoS 在全部 12 种基准、智能体及离线或在线设置下平均提升 7.8 分，并在 11 种设置中平均超出五种现有不确定性触发方法中最强者 2.9 分。消融实验显示，基于实测结果训练与严格伤害预算是其关键。
正文 · 原文
Abstract:Steering an LLM agent means deciding whether to correct it, at which step, and with which mechanism. Uncertainty is often used to decide when to correct an agent, but whether it can guide these decisions remains unclear. We steer agent trajectories separately at every non-terminal step with each of four mechanisms and run each continuation to completion. The resulting stepwise outcome table (SOT) holds about 82,000 counterfactual continuations of 1,864 trajectories from three benchmarks and two agents. It shows that uncertainty can identify failing trajectories, but that no single signal reliably locates the step at which steering helps. We therefore propose VoS (Value of Steering), a trajectory-level monitor, offline or online, that learns from SOT the value of steering at each step and decides where to steer by it. A harm-budgeted trigger decides whether to steer, limiting the fraction of successful trajectories that VoS disturbs. VoS improves on unmodified execution in all 12 settings of benchmark, agent, and offline or online use, by 7.8 points on average, and outperforms the strongest of five existing uncertainty-triggered methods in 11, by 2.9 points on average. Ablations show that training on measured outcomes and a tight harm budget are both essential.
| Comments: | 22 pages, 9 figures, 9 tables |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.09115 [cs.AI] |
| (or arXiv:2610.09115v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09115 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hanwen Li [view email]
[v1]
Tue, 6 Oct 2026 21:05:50 UTC (466 KB)
来源：arXiv cs.CL · arxiv.org