arXiv cs.AI· Simon Ott, Yufan Chen, Ruiping Liu, Junwei Zheng, Jiale Wei, Di Wen, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen·· 3 天前AI 评分26
VL-DocIR：面向视觉长文档信息检索的基准
VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval
AI 导读
VL-DocIR 是一个面向视觉长文档检索的页面级基准，由 Wikipedia、arXiv、PubMed 和 SEC 代理声明的 29,641 篇 HTML 文档渲染为 388,548 张页面图像，包含 23 个领域、271,760 条查询和六种证据结构。
正文 · 原文
Abstract:Vision-based document retrieval has improved rapidly, but current evaluation settings still emphasize short documents and single relevant pages. This leaves an important gap between leaderboard performance and realistic long-document retrieval, where the evidence needed to answer a query may span multiple pages or even multiple documents. We present VL-DocIR, a page-level benchmark for evidence-complete visual long-document retrieval. VL-DocIR is built from 29,641 HTML-born documents rendered into 388,548 page images from Wikipedia, arXiv, PubMed, and SEC proxy statements. It contains 271,760 queries across 23 domains and six evidence structures, covering single-page, within-document multi-page, and cross-document evidence configurations. Evidence annotations are grounded to rendered pages and HTML element identifiers, and queries are filtered with a cleaning pipeline targeting clarity, correctness, closedness, and the absence of explicit layout references. We evaluate leading single-vector and multi-vector visual retrievers together with diagnostic retrieval settings such as document-context scoring, query splitting, reranking, and round-robin ensembles. The strongest evaluated configuration reaches 69.3 nDCGAll@10 and 80.6 RecallAll@10 under structure-balanced Macro evaluation, but source- and evidence-structure breakdowns show that cross-document retrieval remains substantially harder than single-page retrieval, and that document-context scoring is especially helpful for within-document multi-page queries. VL-DocIR exposes these failure modes and provides a more demanding target for visual long-document retrieval. The new dataset and code will be made publicly available at this https URL.
| Comments: | Accepted by NeurIPS 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.17057 [cs.AI] |
| (or arXiv:2610.17057v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17057 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yufan Chen [view email]
[v1]
Mon, 12 Oct 2026 16:25:31 UTC (8,006 KB)
来源：arXiv cs.AI · arxiv.org