arXiv cs.AI· Genshin Kakimoto, Atsushi Yanagisawa·· 12 小时前AI 评分31
8.4B 多模态基础模型在日语金融文档上的分阶段持续适配研究
Staged Continual Adaptation of Multimodal Foundation Models for Japanese Financial Documents
AI 导读
一项研究追踪 8.4B 参数多模态基础模型在日语金融文档上经历三个训练阶段的表现，发现各阶段能力交互并非简单叠加：纯数学蒸馏会覆盖此前习得的金融知识，而数学程序性技能仍保留，领域阶段更多是修复而非新增能力。跳过某阶段的消融实验对领域知识最优，但会放弃数值推理收益，说明各阶段贡献不可分解，最佳 checkpoint 取决于任务。
正文 · 原文
Abstract:Staged post-training of multimodal foundation models chains vision-language alignment, reasoning distillation, and domain tuning, yet how each phase trades capabilities against the others remains under-characterised. Tracking an 8.4B-parameter model from an un-fine-tuned baseline through three training phases on Japanese financial documents (evaluated on five text-input benchmarks), we find the phases interact non-additively: math-only distillation overwrites the financial knowledge acquired earlier even as procedural math skill survives, and the domain phase repairs more than it installs. A skip-phase ablation is best for domain knowledge but forfeits numerical-reasoning gains, so per-phase contributions are not decomposable and the best checkpoint is task-dependent, with implications for checkpoint retention and scheduling rather than added stages.
| Comments: | 17 pages, 10 figures, 9 tables. Presented at the ICML 2026 Workshop "Continual Adaptation at Scale: Towards Sustainable AI". Code: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.27904 [cs.AI] |
| (or arXiv:2610.27904v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.27904 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Atsushi Yanagisawa [view email]
[v1]
Tue, 20 Oct 2026 09:19:36 UTC (1,838 KB)
来源：arXiv cs.AI · arxiv.org