arXiv cs.CL· Rajeel Ahmad Ansari, Tautik Agrahari·· 12 小时前AI 评分34
不确定性何时能衡量 RAG 幻觉风险？研究揭示检索覆盖度的条件性作用
When Does Uncertainty Measure RAG Hallucination Risk?
AI 导读
研究在 HotpotQA 和 SQuAD 上测试四个指令微调模型发现，不确定性仅在检索证据能约束答案时才能衡量 RAG 幻觉风险。按词数分箱匹配忠实与幻觉答案后，仅靠长度的分类器接近随机水平，而困惑度在针对性无支撑编辑上达到 0.81–0.91 AUROC。
正文 · 原文
Abstract:Uncertainty can measure hallucination risk in retrieval-augmented generation (RAG), but only when retrieved evidence constrains the answer. When evidence is present, unsupported factual claims become less likely; when retrieval coverage is poor, the model can answer from internal knowledge and assign high probability to unsupported claims. We test this across four instruction-tuned models on HotpotQA and SQuAD. After matching faithful and hallucinated answers within word-count bins, length-only classifiers fall near chance, while perplexity reaches 0.81--0.91 AUROC on targeted unsupported edits. Varying retrieval depth shows why the score changes meaning: detection can weaken with only the top-ranked passage, then recover after adding a second passage. A combined uncertainty and semantic-drift score improves calibration, and measurements remain above chance on RAGTruth QA. Results support a conditional view of uncertainty as a useful measure of RAG hallucination risk.
| Comments: | 6 pages. Accepted for oral presentation at ICNLSP 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.22615 [cs.CL] |
| (or arXiv:2610.22615v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22615 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Proceedings of the 9th International Conference on Natural Language and Speech Processing (ICNLSP-2026), pp. 53-58, 2026 |
Submission history
From: Rajeel Ahmad Ansari [view email]
[v1]
Thu, 15 Oct 2026 20:17:26 UTC (51 KB)
来源：arXiv cs.CL · arxiv.org