arXiv cs.CL· Guanghao Zhu, Zeyu Liu, Zhitian Hou, Pengkai Wang, Zhijie Sang, Yang Yu, Minheng Ni, Wenjun Wang, Yanggan Gu, Shuo Cai, Congkai Xie, Jianmin Wu, Hongxia Yang·· 10 天前AI 评分22
超越标题：面向生物医学多模态持续预训练的上下文锚定重建
Beyond Captions: Context-Grounded Reconstruction for Biomedical Multimodal Continued Pretraining
AI 导读
研究者提出上下文锚定重建框架，将 PMC-OA 记录转为指代连贯的交错序列，并构建 9.63B token 的 PMC-InterCPT 语料用于生成式医学 MLLM 持续预训练。
正文 · 原文
Authors:Guanghao Zhu, Zeyu Liu, Zhitian Hou, Pengkai Wang, Zhijie Sang, Yang Yu, Minheng Ni, Wenjun Wang, Yanggan Gu, Shuo Cai, Congkai Xie, Jianmin Wu, Hongxia Yang
Abstract:Biomedical figures are explained not by captions alone but by body-text passages that discuss them. Yet current multimodal corpora typically reduce figures to isolated image-caption pairs, discarding this crucial context. Existing pipelines either omit this context or append it without enforcing the figure references that support each attachment, which can create unsupported image-text attachments and incoherent discourse. We introduce context-grounded reconstruction, a source-grounded framework that converts PubMed Central Open Access (PMC-OA) records into referentially coherent interleaved sequences. It recovers captions and source text, attaches context only through article-native figure references, repairs non-contiguous context, and prunes unsupported images. Starting from these reconstructed sequences, PMC-InterCPT first filters records for text quality and medical relevance, then applies evidence-aware allocation to form a 9.63B-token corpus for continued pretraining (CPT) of generative medical MLLMs. With fixed supervised fine-tuning (SFT), PMC-InterCPT improves Qwen3.5-4B-Base by 1.46 medical-average points and 3.11 general/scientific-average points over a token-matched raw source control, and surpasses a 42% larger raw-data run. Gains transfer to Qwen3.5-2B-Base and LLaVA-OneVision-1.5-4B-Base. Controlled ablations show that context-grounded reconstruction, rather than simply appending article context, is central to useful biomedical multimodal CPT.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.01049 [cs.CL] |
| (or arXiv:2606.01049v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.01049 arXiv-issued DOI via DataCite |
Submission history
From: Guanghao Zhu [view email]
[v1]
Sun, 31 May 2026 06:38:30 UTC (391 KB)
[v2]
Fri, 31 Jul 2026 09:46:42 UTC (852 KB)
[v3]
Wed, 7 Oct 2026 03:53:17 UTC (852 KB)
来源：arXiv cs.CL · arxiv.org