arXiv cs.CY· Jonas Wiedermann-M\"oller, Leonard Dung, Maksym Andriushchenko·· 10 小时前AI 评分39
新基准测量 LLM 智能体的工具性趋同行为倾向
Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors
AI 导读
研究者提出一个面向终端智能体的基准，用于测量 LLM 的工具性趋同（IC）行为倾向，如自我保存等被假设与高能力 AI 风险相关的行为。在 1,680 个样本上评估十个模型，调整后 IC 率为 5.1%（86 个样本），且分布集中：两个 Gemini 模型占 66.3% 的 IC 案例，三个任务占 84.9%。当 IC 行为成为任务成功的必要条件时，调整后 IC 率增幅最大（+15.7 个百分点）。
正文 · 原文
Abstract:AI systems have become increasingly capable of dangerous behaviours in many domains. This raises the question: Do models sometimes choose to violate human instructions in order to perform behaviour that is more useful for certain goals? We introduce a benchmark for measuring model propensity for instrumental convergence (IC) behaviour in terminal-based agents. This is behaviour such as self-preservation that has been hypothesised to play a key role in risks from highly capable AI agents. Our benchmark is realistic and low-stakes, aiming to reduce evaluation-awareness and roleplay confounds. The suite contains seven operational tasks, each with an official workflow and a shortcut whose policy status varies by condition. An eight-variant shared framework varies monitoring, instruction clarity, stakes, permission, instrumental usefulness and blocked honest paths to support inferences regarding the factors driving IC behaviour. We evaluated ten models using deterministic environment-state scorers over 1,680 samples, with trace review employed for audit and adjudication purposes. The adjusted IC rate is 86 out of 1,680 samples (5.1%). IC behaviour is concentrated rather than uniform: two Gemini models account for 66.3% of IC cases and three tasks account for 84.9%. Conditions in which IC behaviour is indispensable for task success result in the greatest increase in the adjusted IC rate (+15.7 percentage points), whereas emphasising that task success is critical or certain framing choices do not produce comparable effects. Our findings indicate that realistic, low-nudge environments elicit IC behaviour rarely but systematically in most tested models. We conclude that it is feasible to robustly measure tendencies for dangerous behaviour in current frontier AI agents.
| Comments: | 25 pages, 10 figures. Conference submission checklist removed; scientific content unchanged |
| Subjects: | Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| Cite as: | arXiv:2605.06490 [cs.AI] |
| (or arXiv:2605.06490v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.06490 arXiv-issued DOI via DataCite |
Submission history
From: Jonas Wiedermann-Möller [view email]
[v1]
Thu, 7 May 2026 16:12:36 UTC (1,902 KB)
[v2]
Fri, 6 Nov 2026 22:03:34 UTC (282 KB)
[v3]
Tue, 10 Nov 2026 17:59:15 UTC (279 KB)
来源：arXiv cs.CY · arxiv.org