arXiv cs.AI· Roman Belaire, Arunesh Sinha, Pradeep Varakantham·· 3 天前AI 评分32
用分层强化学习对对话 LLM 进行红队测试
There are Levels to It: Red Teaming Dialogue With Hierarchical Reinforcement Learning
AI 导读
研究者提出一种分层强化学习红队测试范式，将红队测试形式化为马尔可夫决策过程，以最大化多轮交互的预期累积危害。其生成式智能体借助 token 级危害奖励学习多样化多轮攻击，可持续发现绕过基线的 LLM 漏洞，并取得新的 SOTA。该工作被 NeurIPS 2026 收录。
正文 · 原文
Abstract:Red teaming is essential for securing Large Language Models, yet current automated methods remain limited by templates and single-turn attacks. To simulate the complex, interactive nature of real-world adversarial attacks, we introduce a novel red teaming paradigm designed to maximize expected cumulative harm through strategic interaction. By formalizing red teaming as a Markov Decision Process in a hierarchical reinforcement learning framework, we navigate the challenges of sparse rewards and long-horizon planning. Our generative agent learns diverse, multi-turn attacks using a token-level harm reward, consistently uncovering vulnerabilities that bypass baselines. This approach achieves a new state of the art and reframes LLM red teaming as a principled, trajectory-based process.
| Comments: | NeurIPS 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2508.04451 [cs.LG] |
| (or arXiv:2508.04451v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2508.04451 arXiv-issued DOI via DataCite |
Submission history
From: Roman Belaire [view email]
[v1]
Wed, 6 Aug 2025 13:52:00 UTC (2,043 KB)
[v2]
Tue, 13 Oct 2026 01:53:44 UTC (1,893 KB)
来源：arXiv cs.AI · arxiv.org