arXiv cs.AI· Manvendra Modgil·· 6 天前AI 评分28
基于情感的触发器与 LLM 评判为何无法为自主智能体把握干预时机：饱和陷阱与干预时机的主观性
The Saturation Trap and the Subjectivity of Intervention Timing: Why Affect-Based Triggers and LLM Judges Fail to Time Interventions on Autonomous Agents
AI 导读
研究用 18 维情感动力学引擎 HEART 在 SWE-bench-Verified 调试轨迹上评测四类干预触发器，发现状态阈值会因"状态饱和陷阱"在 39-83% 的动作上误触发。
正文 · 原文
Abstract:As autonomous AI agents move from conversational systems to long-horizon software execution, runtime safety layers that decide when to interrupt an agent have become essential. We study this timing problem using a continuous 18-dimensional affective-dynamics engine (HEART) as a diagnostic probe, evaluating four intervention trigger families - absolute state thresholds, composite state-action patterns, regex reasoning-feature extraction, and zero-shot LLM-as-judge - against human-annotated intervention points on SWE-bench-Verified debugging traces. We report three findings. First, a State Saturation Trap: modeled frustration quickly crosses the threshold and stays at its maximum (v2 erratum: the observer could not register recovery; decay never ran), converting threshold-on-state triggers from moment detectors into near-constant indicators that fire on 39-83% of actions across five trajectories. Second, a capability-and-context floor for LLM judges: a small model(gpt-5.4-mini) never fires, while frontier and cross-vendor models escape the zero-firing floor only with full-trajectory context, reaching only F1 0.17-0.40 at up to 90x the cost. Third, and most importantly, the supervised target is not reproducible among humans: three trained annotators using the same rubric on a 56-action trajectory agree on where to intervene only slightly above chance (location Krippendorff's alpha = +0.047; best pairwise Cohen's kappa = +0.349) and not at all on intervention type (pause degenerate; clarify below chance; reflect only alpha = +0.226). We conclude that intervention timing is a low-reliability construct, making single-annotator F1 an unsuitable optimization target. Our contribution is the joint mapping of this problem across human inter-rater reliability, four detector architectures, a cross-model LLM-judge sweep, and a reproduced saturation effect, rather than any single detector's accuracy.
| Comments: | v2: adds an erratum notice (decay term never executed in replays; the observer could not register recovery). Results otherwise unchanged. 12 pages, 5 tables. Code and data: this https URL |
| Subjects: | Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.7; I.2.11; H.5.2 |
| Cite as: | arXiv:2606.04296 [cs.AI] |
| (or arXiv:2606.04296v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.04296 arXiv-issued DOI via DataCite |
Submission history
From: Manvendra Modgil [view email]
[v1]
Tue, 2 Jun 2026 23:54:27 UTC (13 KB)
[v2]
Fri, 9 Oct 2026 06:49:14 UTC (15 KB)
来源：arXiv cs.AI · arxiv.org