跳到正文
arXiv cs.AI· Zhihang Yi, Jian Zhao, Jiancheng Lv, Tao Wang·· 10 小时前AI 评分20

多模态信息融合驱动的图表理解:MLLMs 综述——演进、局限与认知增强

Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancement

AI 导读

一篇综述系统梳理了 MLLMs 在图表理解这一多模态信息融合任务上的研究进展,提出将评测基准划分为 canonical 与 non-canonical 的新分类法,并沿经典深度学习到前沿 MLLM 融合策略的路径追踪方法演进。文章重点剖析当前模型在感知与推理上的缺陷,指出先进对齐技术与强化学习是实现认知增强的潜在方向。

正文 · 原文

View PDF HTML (experimental)

Abstract:Chart understanding is a quintessential information fusion task, requiring the seamless integration of graphical and textual data to extract meaning. The advent of Multimodal Large Language Models (MLLMs) has revolutionized this domain, yet the landscape of MLLM-based chart analysis remains fragmented and lacks systematic organization. This survey provides a comprehensive roadmap of this nascent frontier by structuring the domain's core components. We begin by analyzing the fundamental challenges of fusing visual and linguistic information in charts. We then categorize downstream tasks and datasets, introducing a novel taxonomy of canonical and non-canonical benchmarks to highlight the field's expanding scope. Subsequently, we present a comprehensive evolution of methodologies, tracing the progression from classic deep learning techniques to state-of-the-art MLLM paradigms that leverage sophisticated fusion strategies. By critically examining the limitations of current models, particularly their perceptual and reasoning deficits, we identify promising future directions, including advanced alignment techniques and reinforcement learning for cognitive enhancement. This survey aims to equip researchers and practitioners with a structured understanding of how MLLMs are transforming chart information fusion and to catalyze progress toward more robust and reliable systems.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2602.10138 [cs.CV]
  (or arXiv:2602.10138v2 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2602.10138

arXiv-issued DOI via DataCite

Journal reference: Information Fusion 138 (2027) 104700
Related DOI: https://doi.org/10.1016/j.inffus.2026.104700

DOI(s) linking to related resources

Submission history

From: Zhihang Yi [view email]
[v1] Sun, 8 Feb 2026 12:59:50 UTC (28,457 KB)
[v2] Wed, 18 Nov 2026 06:57:32 UTC (26,693 KB)

来源:arXiv cs.AI · arxiv.org