arXiv cs.CL· Kevin Glocker, K\"atriin Kukk, Romina Oji, Marcel Bollmann, Marco Kuhlmann, Jenny Kunz·· 11 小时前AI 评分27
突破多语言诅咒：高效语言适配的扩展策略
Outgrowing the Curse of Multilinguality: Scaling Strategies for Efficient Language Adaptation
AI 导读
研究发现，从英语基座模型进行参数扩展（upscaling）是提升目标语言性能的高效适配策略，可缓解容量约束，对高资源语言尤其有效。在低资源场景下，先在英语上做初步扩展有益；而在高资源场景下，直接在目标语言上扩展计算效率更高，尽管多数语言的数据效率会略有牺牲。对于多语言持续预训练，直接用目标数据扩展是兼顾数据效率与减少英语遗忘的最优方案。
正文 · 原文
Abstract:Training high-performing language models which include medium- and lower-resource languages remains a challenge. Massively multilingual models still underperform language-specific adaptations, especially at smaller scales, due to the curse of multilinguality. In this work, we investigate scaling as an efficient strategy for adapting pretrained models to new target languages. Through comprehensive ablations matched for data and training FLOPs, we test whether upscaling from English base models enables more effective and resource-efficient adaptation than standard continued pretraining. We find that upscaling is an efficient strategy for improving target-language performance by relieving capacity constraints, especially for higher-resourced languages. An initial upscaling step in English is beneficial for low-resource languages. However, in higher-resource scenarios it is more compute-efficient to directly scale up on the target language, albeit slightly compromising data efficiency for most languages. For multilingual continued pre-training, we identify directly scaling up using the target data as the optimal setup for both data efficiency and reduced English forgetting.
| Comments: | Long paper published in Findings of AACL-IJCNLP 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2512.10772 [cs.CL] |
| (or arXiv:2512.10772v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2512.10772 arXiv-issued DOI via DataCite |
|
| Journal reference: | Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026, pp. 4329-4344 |
Submission history
From: Kevin Glocker [view email]
[v1]
Thu, 11 Dec 2025 16:09:54 UTC (125 KB)
[v2]
Fri, 6 Nov 2026 16:22:36 UTC (230 KB)
来源：arXiv cs.CL · arxiv.org