arXiv cs.CL· Rohan Menon, Chih-Hong Cheng·· 13 小时前AI 评分31
超越攻击成功率：评估迭代式 LLM 越狱的成本感知指标 ASR_STEP
Beyond Attack Success Rate: Cost-Aware Metric for Evaluating Iterative LLM Jailbreaks
AI 导读
研究提出 ASR_STEP 指标，在保留二元成功判定的同时对受害模型的额外调用进行惩罚，以弥补传统攻击成功率（ASR）忽略重复调用的缺陷。在 JailBreakBench 上对 GCG、带步数预算的 PAIR 和单步 PAP 基线跨三个模型族评测，GCG 折扣后分数下降 7.0-8.7 个百分点，PAP 保持不变，PAIR 的折扣区间为 3.4-12 个百分点。
正文 · 原文
Abstract:Attack success rate (ASR) evaluates jailbreaks against large language models without accounting for repeated calls to the victim model. Yet each additional call gives the deployed system another opportunity to detect the attack and intervene. We introduce $\mathrm{ASR}_{\mathrm{STEP}}$, which penalizes additional victim-model calls while retaining binary success judgments. Our evaluation considers GCG, PAIR with a step budget, and a single-step PAP baseline on JailBreakBench across three model families. GCG's scores decrease by 7.0-8.7 percentage points after discounting, while PAP's remain unchanged. While for PAIR attack, we see a wider discounting range between 3.4-12 percentage points, showing that the "step" quantity differs per iterative strategy. Reporting both metrics makes the discount due to repeated victim-model interaction explicit. Code is available here: this https URL.
| Subjects: | Cryptography and Security (cs.CR); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.26589 [cs.CR] |
| (or arXiv:2610.26589v2 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.26589 arXiv-issued DOI via DataCite |
Submission history
From: Rohan Menon [view email]
[v1]
Mon, 19 Oct 2026 15:36:41 UTC (54 KB)
[v2]
Thu, 22 Oct 2026 08:39:36 UTC (54 KB)
来源：arXiv cs.CL · arxiv.org