arXiv cs.AI· Pingyu Wu, Weiming Zhang, Nenghai Yu·· 4 小时前AI 评分58
论文指出 LLM 滥用防护依赖不可信输入,129 篇论文防御方案均存在该漏洞
LLM Misuse Safeguards Rely on Untrusted Input: A Widespread, High-Risk Vulnerability
AI 导读
arXiv 论文指出,LLM 滥用防护机制依据用户自述的职业身份或用途来判断是否给出回答,而任何人都能编造这类声明,攻击者已借此造成实际危害。作者统计最新一届 12 个顶会共 129 篇论文提出的防御方案,发现全部存在这一漏洞,现有防护基准也无法暴露该问题。论文认为漏洞根源在于用户输入本身,并提出一种补丁方案,使模型在帮助专业人士的同时不向冒充者输出危险能力。
正文 · 原文
Abstract:Large language models now help professionals in fields such as cybersecurity and biology, yet the same answers would help an attacker. Deployers rely on misuse safeguards to keep such answers from attackers. We find that these safeguards share a high-risk vulnerability: they decide whether to release an answer from what users write, including the professional identity or purpose they claim, and anyone can write such a claim. Threat reports from model vendors and security firms show that attackers who fabricate a professional identity or purpose already exploit this vulnerability to cause serious harm. The vulnerability is also widespread across the state of the art: every defense proposed in 129 papers from the latest editions of 12 top venues has it. Safeguard research studies the wrong object: it treats harm as a property of the text of a request, although the same text can come from a professional or an attacker. Current safeguard benchmarks therefore cannot reveal the vulnerability: a safeguard can score high on them and still give dangerous answers to attackers. Because the vulnerability lies in what users write, no improvement in how safeguards judge that writing can close it. Under the patch we propose, large language models can help professionals without giving dangerous capabilities to anyone who merely claims to be one.
| Comments: | 13 pages, 4 tables. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible |
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.27951 [cs.CR] |
| (or arXiv:2607.27951v2 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2607.27951 arXiv-issued DOI via DataCite |
Submission history
From: Pingyu Wu [view email]
[v1]
Thu, 30 Jul 2026 09:59:08 UTC (249 KB)
[v2]
Tue, 17 Nov 2026 16:34:38 UTC (124 KB)
来源:arXiv cs.AI · arxiv.org