arXiv cs.CL· Yifan Zhao, Qianyang Zhao, Xinyu Guan, kai wei, Yuming Deng·· 13 小时前AI 评分34
QESChunker：用单一目标统一 RAG 的重叠与非重叠分块
QESChunker: A Single Objective Unifies Overlapping and Non-Overlapping Chunking for RAG
AI 导读
QESChunker 是首个利用问题—证据监督构建静态分块索引的方法，通过统一的问题—证据效用最大化目标，将重叠与非重叠分块转化为监督式组合优化问题。非重叠约束下可用动态规划在多项式时间内精确求解，允许重叠时目标为单调子模，贪心算法达到 (1-1/√e) 近似。在两个自建和三个公开数据集上，两种变体在相同重叠设置与固定混合检索器下，几乎全部评测设置的完整证据召回率均优于现有分块方法。
正文 · 原文
Abstract:Chunking is a fundamental preprocessing step in retrieval-augmented generation (RAG), yet most existing chunkers rely primarily on heuristics. We introduce the Question--Evidence-Supervised Chunker (QESChunker), to our knowledge the first method for constructing static chunk indices from question--evidence supervision. QESChunker constructs document-specific chunk indices by optimizing over human-annotated or LLM-generated question--evidence pairs, without access to held-out test queries. We formulate non-overlapping and overlapping index construction through a unified question--evidence utility-maximization objective, turning chunking into a supervised combinatorial optimization problem. Under the non-overlap constraint, the objective decomposes over chunks and can be optimized exactly in polynomial time by dynamic programming. When overlap is allowed, the objective becomes monotone submodular; exact optimization is NP-hard, but our greedy achieves a $(1-1/\sqrt{e})$-approximation. Across two self-constructed and three public datasets, both variants outperform existing chunking methods in complete-evidence recall in nearly all evaluated settings, under the same overlap regime and a fixed hybrid retriever. These results recast chunking as supervised, budget-aware index construction for evidence retrieval.
| Comments: | 16 pages, 8 figures, 4 tables |
| Subjects: | Information Retrieval (cs.IR); Computation and Language (cs.CL); Optimization and Control (math.OC) |
| Cite as: | arXiv:2610.36557 [cs.IR] |
| (or arXiv:2610.36557v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36557 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Qianyang Zhao [view email]
[v1]
Tue, 27 Oct 2026 03:05:09 UTC (303 KB)
来源：arXiv cs.CL · arxiv.org