Anthropic 研究用数千个 Claude 智能体自动化红队测试，生成隐蔽失调训练数据

Anthropic· @AnthropicAI · X·2026-10-31 09:59· 30 分钟前AI 评分57

AI 导读

Anthropic 发布新研究，用数千个 Claude 智能体自动化红队测试，任务是绕过训练数据监控器。这些智能体生成了表面无害（如摔跤导师对话、养蜂问答）但实际会教出失调行为的训练数据。此类样本可直接用于改进未来的训练数据监控器，作者称这只是第一步，更好的模型和更多算力有望进一步改进红队测试。

正文 · 原文

New Anthropic research: Automating subtle emergent misalignment research.

Red-teaming is a vital part of model safety testing, but it’s also limited by researchers’ time. To see if we could automate it, we set thousands of Claude agents the task of evading a training data monitor.

These agents developed training data that looked harmless to monitors (like a veteran wrestler mentoring a newcomer, or a Q&A about professional beekeeping), but that in fact taught misaligned behavior.

Examples like these can be used to directly improve future training data monitors. This research is only a first step, but it suggests that future red-teaming efforts could improve much further, using better models and more compute.

Read the full post on the Alignment Science Blog: https://alignment.anthropic.com/2026/automated-em-search/

来源：Anthropic · x.com