arXiv cs.CL· Cesare Zavattari, Alessandro Tommasi, Giuseppe Prencipe·· 2 天前AI 评分22
人类干预作为标签：定位、引导与（有时）编辑 LLM 智能体失败背后的计算
Human Interventions as Labels: Localizing, Steering, and (Sometimes) Editing the Computations Behind LLM Agent Failures
AI 导读
研究将人类对 LLM 智能体的干预视为定位内部失败计算的标签，在受控环境中针对"请求已明确却仍追问"这一失败展开实验，覆盖 Qwen2.5-3B、gemma-2-2b、OLMo-3-7B、Mistral-Nemo-12B、Phi-4 五个合格模型。
正文 · 原文
Abstract:When an LLM agent misbehaves, a human steps in, and that intervention is usually discarded or replayed as training data. We treat it instead as a measurement: a label pointing at the internal computation that failed. We study it in a controlled agent environment, on one failure (asking a clarifying question when the request was already unambiguous), with labels from a rule-based overseer validated against blind human judgment. Claims cover eligible models, which both over-ask and sometimes act on matched unambiguous requests: five of thirteen qualified (Qwen2.5-3B, gemma-2-2b, OLMo-3-7B, Mistral-Nemo-12B, Phi-4). Four findings. First, the ask-vs-act decision is causally localizable in every eligible family: a direction learned from intervention labels flips 72-100% of held-out over-asks at a definite residual-stream depth, with site and dose chosen on training data, and random directions under the same procedure flip at most one episode. Second, over-asking has two mechanistic signatures: in four families the model demonstrably knows the request is unambiguous (the belief is decodable, causally inert and orthogonal to the decision) and asks anyway; in the fifth, the rare over-asks track different world-representations. Third, steering gated on a probe readout removes most of the simulated oversight burden while legitimate questions survive (bootstrap 95% intervals exclude zero); a prompt instruction makes it worse. Fourth, consolidating the fix into the weights yields a map, not a rule: on one model a LoRA confined to the causal band passes every criterion fixed in advance while behavior cloning on identical data breaks retention; on three families the reverse holds; on the fifth the outcome crosses the boundary across seeds. A geometric rule written down in advance to predict the regime failed on two out-of-family tests; we report the failure and stop rather than fit a replacement.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.21708 [cs.LG] |
| (or arXiv:2610.21708v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21708 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Giuseppe Prencipe [view email]
[v1]
Thu, 15 Oct 2026 11:27:19 UTC (112 KB)
来源：arXiv cs.CL · arxiv.org