arXiv cs.CL· Zichun Guo, Yuling Shi, Wenhao Zeng, Chao Hu, Haotian Lin, Terry Yue Zhuo, Jiawei Chen, Xiaodong Gu, Wenping Ma·· 13 小时前AI 评分31
ShredBench：评估多模态 LLM 在文档重建中的语义推理能力
ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstruction
AI 导读
研究者提出 ShredBench 基准，用自动化流水线从 Markdown 直接生成碎片化文档，评估多模态 LLM 的文档重建能力。该基准覆盖英文、中文、代码、表格四种场景，设 8、12、16 三种碎片粒度，可灵活接入最新文本源以避免训练数据污染。对 SOTA 多模态 LLM 的评测显示，文档完整时方法有效，但碎片化后重建难度骤增，NED 随碎片数量增加而急剧下降。
正文 · 原文
Abstract:Multimodal Large Language Models (MLLMs) have achieved remarkable performance in Visually Rich Document Understanding (VRDU) tasks, but their capabilities are mainly evaluated on pristine, well-structured document images. We consider content restoration from shredded fragments, a challenging VRDU setting that requires integrating visual pattern recognition with semantic reasoning under significant content discontinuities. To facilitate systematic evaluation of complex VRDU tasks, we introduce ShredBench, a benchmark supported by an automated generation pipeline that renders fragmented documents directly from Markdown. The proposed pipeline ensures evaluation validity by allowing the flexible integration of latest or unseen textual sources to prevent training data contamination. ShredBench assesses four scenarios (English, Chinese, Code, Table) with three fragmentation granularities (8, 12, 16 pieces). Empirical evaluations on state-of-the-art MLLMs reveal a significant performance gap: The method is effective on intact documents; however, once the document is shredded, restoration becomes a significant challenge, with NED dropping sharply as fragmentation increases. Our findings highlight that current MLLMs lack the fine-grained cross-modal reasoning required to bridge visual discontinuities, identifying a critical gap in robust VRDU research.
| Comments: | ACL 2026 Findings. Code available at this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2604.23813 [cs.CV] |
| (or arXiv:2604.23813v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2604.23813 arXiv-issued DOI via DataCite |
Submission history
From: Yuling Shi [view email]
[v1]
Sun, 26 Apr 2026 17:26:06 UTC (801 KB)
[v2]
Tue, 27 Oct 2026 17:24:02 UTC (805 KB)
来源：arXiv cs.CL · arxiv.org