跳到正文
arXiv cs.AI· Keshav Shenoy, Anthony Zang, Eric Gan, Jacqueline Tay, Amy Ngo, Sebastian Prasanna, Julian Stastny, Vivek Hebbar·· 3 小时前AI 评分58

研究:审计员难以从AI安全研究完整记录中识别蓄意破坏

Auditors Struggle to Catch Sabotage In End-to-End Transcripts of AI Safety Research

AI 导读

一项 arXiv 论文用人类专家扮演破坏者开展红蓝对抗,审计员在获得完整研究记录并可运行实验的情况下,区分被破坏与正常研究的 AUROC 仅为 0.59,与随机猜测无显著差异,仅 17% 的案例能完整指出具体破坏方式。

正文

View PDF HTML (experimental)

Abstract:As AI systems are increasingly used to automate AI R&D, misaligned systems could sabotage research intended to improve model safety and alignment. To stress-test defenses against such sabotage, we use a human expert as a proxy saboteur on AI safety research. We conduct a red-team/blue-team game where a human researcher runs AI safety experiments from scratch and auditors try to tell which were sabotaged and how. Given access to the full research transcript and the ability to run experiments, human auditors mostly fail to distinguish sabotaged from benign research: they achieve an AUROC of 0.59, not significantly better than chance, and fully identify the specific sabotage in only 17% of cases. Despite not being allowed to run experiments, the best frontier models outperform human auditors (Fable 5.1: AUROC 0.73, 32% full identification) within noise and weaker models perform comparably. When the saboteur chooses which experiments are attacked, rather than being required to sabotage every question given, sabotage identification drops further. Our results suggest that research sabotage by expert-level agents is difficult to detect with current auditing methods and provide a new setting for studying research sabotage.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.21832 [cs.AI]
  (or arXiv:2611.21832v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.21832

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Keshav Shenoy [view email]
[v1] Mon, 16 Nov 2026 19:52:38 UTC (669 KB)

来源:arXiv cs.AI · arxiv.org