arXiv cs.AI· Xurui Li, Kaisong Song·· 4 小时前AI 评分31
MACE:MCTS 引导的对抗共进化框架,用于 LLM 安全对齐
From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
AI 导读
研究者提出 MACE,一个 MCTS 引导的对抗共进化框架,将搜索转化为攻击方与防御方的策略学习信号。其 GS-MCTS 组件探索多步改写算子组合,AC-TGPO 组件利用成功频率与路径深度做难度感知的组相对信用分配。在六个模型-数据集设定中,攻击方 ASR-LR 全部最高,防御方将平均 ASR-LR 从最佳基线的 11.9% 降至 8.8%。
正文 · 原文
Abstract:Large Language Models (LLMs) increasingly power web-facing applications, including conversational assistants and content services, where open user interaction exposes them to diverse safety threats and adaptive attacks. In particular, jailbreak attackers repeatedly reformulate harmful requests to bypass safety alignment. Although existing studies have concentrated on defending dynamic jailbreaks, they often retain discovered attacks as flat samples and discard information about how those attacks were found. In this paper, we introduce MACE, an MCTS-guided adversarial co-evolution framework that turns search into policy-learning signals for both an attacker and a defender. Its search component, GS-MCTS, explores multi-step compositions of rewriting operators and associates discovered attacks with empirical success frequency and path depth. Its optimization component, AC-TGPO, uses these statistics for difficulty-aware group-relative credit assignment and constructs a temporally asymmetric curriculum from current interactions, pairings between the latest attacker and preceding defender, and attacks that survive a defense update. Thus, MACE connects search structure to both within-round policy optimization and cross-round memory, rather than using MCTS only to generate attacks or treating each round as a flat adversarial dataset. Removing difficulty-aware weighting, asymmetric samples, or persistent hard samples increases defense ASR-LR by 3.7 %, 6.0 %, and 5.0 %, respectively. Across evaluated models, the resulting attacker achieves the highest ASR-LR in all six model--dataset settings, while the defender reduces average ASR-LR from 11.9 % for the best baseline to 8.8 %.
| Comments: | Accepted by WSDM, to be published, update arXiv, version |
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2511.19218 [cs.CR] |
| (or arXiv:2511.19218v3 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2511.19218 arXiv-issued DOI via DataCite |
|
| Related DOI: | https://doi.org/10.1145/3862322.3863664
DOI(s) linking to related resources |
Submission history
From: Xurui Li [view email]
[v1]
Mon, 24 Nov 2025 15:23:41 UTC (1,042 KB)
[v2]
Wed, 26 Nov 2025 15:12:18 UTC (1,041 KB)
[v3]
Tue, 17 Nov 2026 04:10:48 UTC (7,622 KB)
来源:arXiv cs.AI · arxiv.org