跳到正文
arXiv cs.AI· Pingyu Wu, Weiming Zhang, Nenghai Yu·· 4 小时前AI 评分52

论文质疑 LLM 防护评测:拦截成功是否等于系统更安全

The Safeguard Worked. Is the LLM System Safer?

AI 导读

Pingyu Wu 等人在论文中提出一个框架,把已发表的防护评测结果换算成在特定结果与威胁模型下仍可能残留的最小和最大有害协助量。作者分析了 52 篇论文中的 85 项防护评测,其中只有 5 项论证了所有攻击路径都经过检查点,4 项提供了拦截后残留内容的证据。在 NeMo Guardrails 默认流式配置的测试中,检查点拦截了 94% 的密码泄露,但无论是否拦截,客户端都收到了密码。

正文 · 原文

View PDF HTML (experimental)

Abstract:Safeguards for large language model services are typically evaluated by their attack-blocking rates on test suites. Deployment decisions also depend on how much harmful assistance attackers can still obtain from the system, which these rates do not measure. To determine what published evaluations establish about deployment safety, we develop a framework that translates reported results into the minimum and maximum harmful assistance that can remain under a specified outcome and threat model. For safeguards that block attacks through a check, we show that the most an attacker can obtain depends on three properties: whether attack paths pass through the check, its miss rate against every attacker in the threat model, and how much assistance remains after a block. We apply this framework to 85 safeguard evaluations from 52 papers. Of these, only five argue that every attack path passes through the check, and four provide evidence on what remains after a block. These evidence gaps have practical consequences: in NeMo Guardrails' default streaming configuration, the check blocked 94% of password leaks in our test, yet the client received the password in every leak, blocked or not. We identify which of the three properties each safeguard category's evaluations omit, and what to measure for each of four ways attacks bypass a check. Deployments that require these measurements before launch can choose and configure safeguards that limit what attackers obtain from the deployed system.
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
Cite as: arXiv:2609.00519 [cs.CR]
  (or arXiv:2609.00519v2 [cs.CR] for this version)
  https://doi.org/10.48550/arXiv.2609.00519

arXiv-issued DOI via DataCite

Submission history

From: Pingyu Wu [view email]
[v1] Tue, 1 Sep 2026 00:38:35 UTC (161 KB)
[v2] Tue, 17 Nov 2026 14:26:46 UTC (101 KB)

来源:arXiv cs.AI · arxiv.org