arXiv cs.CL· Chahana Dahal, Zuobin Xiong·· 11 小时前AI 评分38
研究指出 ASR 指标掩盖 RAG 防御盲区：被阻止不等于被保护
Prevented is Not Protected: What Attack Success Rate Hides about RAG
AI 导读
一项研究提出 Aligned RAG Evaluation 匹配查询协议，联合考察有无防御下的干净与投毒响应，以区分“被阻止的失败”与“被引入的失败”。在 HotpotQA 和 Natural Questions 上的实验显示，现有防御虽能阻止部分攻击，却会引入意外的目标正向回答，且评估标准会改变防御排名。研究认为仅靠攻击成功率（ASR）评估 RAG 投毒防御存在关键盲区。
正文 · 原文
Abstract:Retrieval-Augmented Generation (RAG) is vulnerable to knowledge poisoning, where adversaries manipulate retrieved evidence to influence model outputs. Existing defenses are primarily evaluated using attack success rate (ASR), which obscures defense-induced failures and the mechanisms behind apparent attack prevention. To address this issue, this work introduces \textbf{Aligned RAG Evaluation}, a matched-query protocol that jointly examines clean and poisoned responses, with and without defense. By combining matched conditions analysis, LLM-judge endorsement, and retrieval dependence controls, the proposed framework distinguishes prevented from introduced failures, target mentions from LLM endorsement, and single-passage influence from evidence composition. Experiments on HotpotQA and Natural Questions reveal that existing defenses can introduce unexpected target-positive answers, although they can prevent some attacks, while evaluation criteria can also change defense rankings. Our findings expose critical blind spots in ASR-based evaluation and motivate a more comprehensive assessment of RAG poisoning defenses. The code and the evaluation artifact are available at this https URL.
| Subjects: | Cryptography and Security (cs.CR); Computation and Language (cs.CL); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2610.40742 [cs.CR] |
| (or arXiv:2610.40742v1 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40742 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chahana Dahal [view email]
[v1]
Thu, 29 Oct 2026 15:21:32 UTC (396 KB)
来源：arXiv cs.CL · arxiv.org