arXiv cs.CL· Ankit Gupta·· 9 小时前AI 评分40
NONSENSE 探针揭示稠密检索的语义性缺口
NONSENSE: Non-Semantic Strings Expose Semanticity Gaps in Dense Retrieval
AI 导读
NONSENSE 是一种黑盒稀疏编辑探针,在禁止复制查询词的前提下,仅优化查询与字符串的稠密相似度,构造出明显无语义的字符串。在五个 BEIR 风格数据集和三种主流单向量检索器上,这些字符串经常得分超过检索器自身判定最相关的段落,并常在完整语料中排名第一。结果表明,所评估的稠密检索器存在语义性缺口:最近邻地位可以与段落式语义分离。
正文 · 原文
Abstract:Dense retrieval is central to modern search, question answering, and retrieval-augmented generation, where high similarity is often treated as evidence of semantic relevance. We examine how far this assumption can be stressed. NONSENSE is a black-box sparse-edit probe that constructs visibly non-semantic strings while forbidding query-term copying, optimizing only the dense similarity between a query and the string. Across five BEIR-style datasets and three popular single-vector retrievers, these strings frequently outscore the retriever's own best judged query-relevant passage (QRP), and often rank first in the full corpus. The result indicates a semanticity gap in the evaluated dense retrievers: nearest-neighbor status can be separated from passage-like meaning. These findings highlight the importance of evaluating dense retrievers not only by whether they retrieve relevant documents, but also by how reliably their similarity scores reflect semantic evidence.
| Comments: | Accepted to EACL 2027 (Main Conference). 15 pages |
| Subjects: | Information Retrieval (cs.IR); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.22353 [cs.IR] |
| (or arXiv:2611.22353v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22353 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ankit Gupta [view email]
[v1]
Tue, 17 Nov 2026 08:04:40 UTC (432 KB)
来源:arXiv cs.CL · arxiv.org