跳到正文
arXiv cs.CL· Zhuchenyang Liu, Yao Zhang, Yu Xiao·· 9 小时前AI 评分34

无需坐标或区域标注:视觉文档理解中的证据归因新方法

Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels

AI 导读

研究提出用“文本引用+多模态检索”的语言接口替代坐标接口,在双语 CiteVQA 子集上测试六个开源视觉语言模型,证据召回率从最高 8 点提升至 26-47,归因幻觉率约减半,答案质量基本不变。基于同一引用-检索流程,作者用 GRPO 训练无需区域标注的模型,将 8B 骨干的严格归因准确率从 22.4 提升至 33.8。

正文 · 原文

View PDF HTML (experimental)

Abstract:Reliable visual document understanding requires a model to attribute each answer to the evidence regions that support it. Recent benchmarks and systems express this step through a coordinate interface: the model outputs the coordinates of bounding boxes that mark the evidence regions in the document. Under this interface, vision-language models often fail to identify the right regions even when the answer is correct, a failure known as Attribution Hallucination. We present a study that investigates whether this failure is partially limited by what the model can express through coordinates. On a verified bilingual CiteVQA subset, we compare the coordinate interface with a language interface in which the model outputs only text, quoting its evidence verbatim, and a multimodal retriever returns the location of each quote as a page region proposed by a layout parser (tables and figures are quoted through their captions or notes); the comparison is repeated over six open vision-language models. Compared with the coordinate interface, evidence recall rises from at most 8 points to between 26 and 47 and the hallucination rate roughly halves, with little change in answer quality. Building on this comparison, we use the same quote-and-retrieve pipeline as a training scaffold: because region-level evidence labels are expensive to collect for long documents, we introduce a GRPO recipe whose reward is a judge's reading of the gold answer and crops of the retrieved regions, training the model to quote better evidence without any region labels and raising an 8B backbone's strict attributed accuracy from 22.4 to 33.8. These findings indicate a practical path to improve attribution without a coordinate interface and without costly region-level supervision.
Comments: Accepted to EACL 2027 (main conference). Code: this https URL
Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Information Retrieval (cs.IR)
Cite as: arXiv:2607.24651 [cs.CV]
  (or arXiv:2607.24651v2 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2607.24651

arXiv-issued DOI via DataCite

Submission history

From: Zhuchenyang Liu [view email]
[v1] Mon, 27 Jul 2026 16:49:36 UTC (3,789 KB)
[v2] Tue, 17 Nov 2026 16:42:58 UTC (4,192 KB)

来源:arXiv cs.CL · arxiv.org