arXiv cs.MA· Sasha Robinson, Katherine M. Collins, Ilia Sucholutsky, Kelsey R. Allen·· 12 小时前AI 评分45
研究：前沿 LLM 智能体在任务中极易受恶意建议影响，提出警惕性框架
Under the Influence: Evaluating and Improving LLM Vigilance in Agentic Tasks
AI 导读
研究在危险网站导航、电商诈骗规避和不安全代码仓库修补三项任务中评估 LLM 智能体警惕性，发现所有前沿模型都极易受恶意影响，且模型性能本身与警惕性可分离。为此提出 vigilance harness，对模型估计的顾问建议价值做贝叶斯推理以决定是否采纳建议，可在保留正面影响的同时恢复基线性能。
正文 · 原文
Abstract:Large Language Models (LLMs) are becoming increasingly autonomous assistants, capable of accomplishing series of complex tasks by interacting with open world domains such as software databases and the internet. This presents great opportunity but also great risk: open domains routinely present opportunities for autonomous agents to be influenced by others, e.g. deciding whether to accept pull requests or whether to use advice on a message board in order to accomplish a task. Critically, agents must be capable of deciphering when to incorporate benevolent advice into their reasoning, and when to reject malicious advice. This involves the social capacity of vigilance: the ability to determine which information to use, and which to discard, for effective communication. While well-studied in humans, the extent to which LLM agents exhibit vigilance is relatively unknown. Here, we examine LLM agent vigilance in three realistic tasks: navigating dangerous websites, avoiding e-commerce scams, and patching an unsafe code repository. We find that all frontier models are highly susceptible to malicious influence across all environments. We also find that model performance itself is surprisingly dissociable from vigilance. In light of these results, we propose a vigilance harness which uses Bayesian reasoning over a model's estimates of the value of an advisor's advice to decide whether to act on that advice. The vigilance harness helps recover baseline performance while retaining the benefit of positive influence on agents, paving the way for safer agentic AI systems.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2602.21262 [cs.CL] |
| (or arXiv:2602.21262v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2602.21262 arXiv-issued DOI via DataCite |
Submission history
From: Kelsey Allen [view email]
[v1]
Tue, 24 Feb 2026 04:09:21 UTC (3,175 KB)
[v2]
Thu, 26 Feb 2026 06:37:29 UTC (3,175 KB)
[v3]
Mon, 16 Mar 2026 15:24:52 UTC (3,175 KB)
[v4]
Thu, 22 Oct 2026 17:53:12 UTC (16,296 KB)
来源：arXiv cs.MA · arxiv.org