arXiv cs.AI· Gosha Kozhevnikov, Tyler Tracy·· 2 天前AI 评分48
Screwtape：用对抗性转录编辑对监控器进行红队测试
Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming
AI 导读
研究团队提出 Screwtape 流水线，用 LLM 攻击者编辑真实智能体转录，使 Claude Code 和 Codex 的智能体产生指定危害（如泄露 API key 或生成未被监控的智能体），同时规避仅检测动作的监控器。在 94 份转录、252 次运行中，它生成 35 个通过验证器且逃过自建监控器的攻击，其中 6 个经人工确认有效，还逃过了某前沿开发者提供的留出监控器。
正文 · 原文
Abstract:We present Screwtape, a pipeline that converts real agent transcripts into attacks that evade action-only monitors. An LLM attacker edits a transcript so that the agent causes a specified harm, such as leaking an API key or spawning an unmonitored agent. The edited actions are never executed, so we use an LLM verifier to reject attacks that are inconsistent with the original transcript or below a severity bar. The attacker revises the edit until the monitor no longer flags it. We applied Screwtape to 94 Claude Code and Codex transcripts from Redwood Research staff, optimizing against our own monitors. Across 252 runs, it produced 35 attacks that passed the verifier and evaded these monitors. Six of them, confirmed valid by human review, also evaded a held-out monitor provided by a frontier developer. We therefore demonstrate that a red-teamer without access to a developer's infrastructure can find confirmed failures of the developer's monitor. We expect to find more attacks when the pipeline runs inside the developer's infrastructure, with frontier attacker models that lack refusal safeguards and direct optimization against the production monitor. Developers can use these attacks to fix monitor blind spots, and evaluators can use them as counterexamples to safety claims about the monitor.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.20941 [cs.CR] |
| (or arXiv:2610.20941v1 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20941 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tyler Tracy Mr. [view email]
[v1]
Wed, 14 Oct 2026 21:08:21 UTC (132 KB)
来源：arXiv cs.AI · arxiv.org