arXiv cs.CL· Pasha Abrishamchian, Eugene Yang, Andrew Yates, Nazli Goharian, Ophir Frieder·· 13 小时前AI 评分32
CoverageBench-Negate：面向否定感知检索的覆盖度基准
Coverage Under Contradiction: A Benchmark for Negation-Aware Retrieval
AI 导读
研究者构建了 CoverageBench-Negate，一个在 CoverageBench 基础上加入受控矛盾信息的覆盖度评测基准，覆盖六个数据集、约 63,000 对原始与编辑证据。基线实验中 BM25 区分原始与编辑近孪生证据的能力明显优于稠密检索器（Qwen3 和 Jina），后者接近随机水平。基准已随构建材料在 Hugging Face Datasets 发布。
正文 · 原文
Abstract:Coverage-evaluation benchmarks quantify whether retrieved evidence covers the different pieces of information needed to satisfy an information need. Existing benchmarks mostly consider factual claims that may already be known to pretrained retrieval components, leaving open whether retrievers find corpus evidence when it contradicts those priors. We study how retrievers behave when their parametric knowledge and the information in a corpus disagree. To do so, we develop a process to augment a nugget-based coverage benchmark with controlled contradictions. For each nugget, we create a negated counterpart and edit its supporting documents to support that negation while preserving the context. Applying this process to CoverageBench, a benchmark measuring information coverage, we create CoverageBench-Negate, spanning six datasets and roughly 63,000 original-edited evidence pairs (one per nugget-document link). We attach rich construction metadata to every pair and additionally provide out-of-family LLM judgments over all pairs and a human audit of a sample. In baseline experiments, BM25 separates original from edited near-twin evidence substantially better than dense retrievers (Qwen3 and Jina), which stay near chance. We release the benchmark on Hugging Face Datasets (nuggets, documents, pairs) together with the construction materials, including the prompts used to create it.
| Comments: | 5 pages, 1 figure. Accepted at CIKM 2026 |
| Subjects: | Information Retrieval (cs.IR); Computation and Language (cs.CL) |
| ACM classes: | H.3.3 |
| Cite as: | arXiv:2610.37162 [cs.IR] |
| (or arXiv:2610.37162v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37162 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Pasha Abrishamchian [view email]
[v1]
Tue, 27 Oct 2026 12:54:48 UTC (600 KB)
来源：arXiv cs.CL · arxiv.org