LongHisDoc：中文长历史文档理解综合基准发布

View PDF HTML (experimental)

Abstract:Chinese historical documents are invaluable cultural treasures, but their complexity poses significant comprehension challenges for non-specialists. Despite the success of Large Language Models (LLMs) and Large Vision-Language Models (LVLMs) in modern document tasks, their capabilities in Long (full-book) Historical Document understanding remain largely unexplored. To bridge this gap, we introduce LongHisDoc, the first benchmark tailored for long historical document understanding. LongHisDoc comprises 1,012 expert-annotated QA pairs derived from 101 PDF documents, with an average of 85.1 pages and 42,776.4 tokens per document. It spans 10 categories and features four QA types: single-page, cross-page, multi-hop, and unanswerable. Answering these questions requires integrating multimodal evidence from text, layout, and visual elements. Our evaluation reveals that the top-performing LVLM and LLM achieve only 62.9% and 69.9% accuracy, respectively, highlighting a substantial performance gap. Furthermore, we assess various modal retrievers and the end-to-end RAG performance to provide references for Retrieval-Augmented Generation (RAG) systems. We believe that LongHisDoc serves as a comprehensive benchmark to catalyze future research in full-book historical document understanding. The benchmark and evaluation code are available at this https URL and this https URL.

Subjects:	Computation and Language (cs.CL)
Cite as:	arXiv:2610.14531 [cs.CL]
	(or arXiv:2610.14531v2 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.14531 arXiv-issued DOI via DataCite

Submission history

From: Zeyu Shan [view email]
[v1] Sat, 10 Oct 2026 03:44:08 UTC (25,062 KB)
[v2] Tue, 20 Oct 2026 09:06:09 UTC (25,063 KB)