arXiv cs.CL· Isotta Magistrali, Chen Shani·· 12 小时前AI 评分38
单独对齐、集体失守：预测 LLM 智能体群体中的对抗性捕获
Aligned Alone, Misaligned Together: Forecasting Adversarial Capture in LLM Agent Populations
AI 导读
研究在安全分诊任务中发现，单独判断几乎一致的 LLM 智能体，在群体中面对持续主张同一决策的“坚定少数派”时会产生截然不同的集体结果，个体审计无法可靠预测群体行为。
正文 · 原文
Abstract:AI safety evaluations typically assess models one at a time. But language-model agents increasingly operate in populations, where they read and respond to one another's decisions. A model that makes reliable decisions on its own may behave differently when surrounded by other agents. Can individually reliable agents be collectively manipulated, and can we predict how strongly the population will respond? We study this question in a security-triage task, where populations of LLM monitors decide whether to escalate or dismiss alerts. We introduce a committed minority of agents that consistently advocates for one decision and measure how its presence shifts the population's behavior. We find that alerts judged almost identically by an isolated agent can produce sharply different collective outcomes. Individual audits therefore cannot reliably predict population behavior. Yet collective vulnerability is not unpredictable: using only benign, adversary-free behavior, we calibrate a response function that forecasts how strongly a population will shift under a future attack, before the attack is run. We next examine how interaction and time shape these outcomes. Allowing agents to observe one another's reasoning can weaken adversarial influence, but does not reliably prevent capture. However, capture need not be permanent. After adversaries are removed or replaced, mean population behavior returns toward its original level, even after five or thirty rounds of pressure. These findings are consistent with a calibrated single-stable-state model, in which adversarial pressure shifts the operating point without creating a second self-sustaining state. The same calibration predicts attack responses in populations of 24 and 48 agents. Alignment in isolation does not guarantee alignment in a population. But a population's behavior before an attack can reveal how it will respond when one arrives.
| Comments: | Accepted as an oral at the NeurIPS 2026 Workshop on Dynamic Alignment in Human-AI Coupled Systems (BiAlign) and as a poster at the NeurIPS 2026 SocialAgent Workshop (Second Workshop on Large Language Models for Social Reasoning and Simulation) |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.22444 [cs.CL] |
| (or arXiv:2608.22444v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.22444 arXiv-issued DOI via DataCite |
Submission history
From: Isotta Magistrali [view email]
[v1]
Sun, 23 Aug 2026 14:49:59 UTC (1,210 KB)
[v2]
Thu, 27 Aug 2026 12:35:05 UTC (1,236 KB)
[v3]
Fri, 30 Oct 2026 22:00:35 UTC (1,457 KB)
来源：arXiv cs.CL · arxiv.org