跳到正文
arXiv cs.CL· Ankit Gupta·· 9 小时前AI 评分40

NONSENSE 探针揭示稠密检索的语义性缺口

NONSENSE: Non-Semantic Strings Expose Semanticity Gaps in Dense Retrieval

AI 导读

NONSENSE 是一种黑盒稀疏编辑探针,在禁止复制查询词的前提下,仅优化查询与字符串的稠密相似度,构造出明显无语义的字符串。在五个 BEIR 风格数据集和三种主流单向量检索器上,这些字符串经常得分超过检索器自身判定最相关的段落,并常在完整语料中排名第一。结果表明,所评估的稠密检索器存在语义性缺口:最近邻地位可以与段落式语义分离。

正文 · 原文

View PDF HTML (experimental)

Abstract:Dense retrieval is central to modern search, question answering, and retrieval-augmented generation, where high similarity is often treated as evidence of semantic relevance. We examine how far this assumption can be stressed. NONSENSE is a black-box sparse-edit probe that constructs visibly non-semantic strings while forbidding query-term copying, optimizing only the dense similarity between a query and the string. Across five BEIR-style datasets and three popular single-vector retrievers, these strings frequently outscore the retriever's own best judged query-relevant passage (QRP), and often rank first in the full corpus. The result indicates a semanticity gap in the evaluated dense retrievers: nearest-neighbor status can be separated from passage-like meaning. These findings highlight the importance of evaluating dense retrievers not only by whether they retrieve relevant documents, but also by how reliably their similarity scores reflect semantic evidence.
Comments: Accepted to EACL 2027 (Main Conference). 15 pages
Subjects: Information Retrieval (cs.IR); Computation and Language (cs.CL)
Cite as: arXiv:2611.22353 [cs.IR]
  (or arXiv:2611.22353v1 [cs.IR] for this version)
  https://doi.org/10.48550/arXiv.2611.22353

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Ankit Gupta [view email]
[v1] Tue, 17 Nov 2026 08:04:40 UTC (432 KB)

来源:arXiv cs.CL · arxiv.org