arXiv cs.CL· Nikit Srivastava, Joel Dag, Ren\'e Speck, Axel-Cyrille Ngonga Ngomo·· 23 小时前AI 评分34
MoE-LoRA 方法用于多语言适配:一项系统性实证研究
MoE--LoRA Methods for Multilingual Adaptation: A Systematic Empirical Study
AI 导读
一项系统性实证研究在冻结基座模型上对比了 8 种 LoRA 方法,覆盖 200 种语言、146 次训练运行和超过 10,000 次评测。结果显示 LoRA 能稳定提升翻译数据的语言模型拟合度、持续降低阅读理解表现,且对低资源语言收益最大;小规模搜索可找到可行配置,但无法可靠预测其相对性能。研究据此认为 LoRA 应被视为有针对性的工具,而非提升 LLM 多语言能力的通用手段。
正文 · 原文
Abstract:Routed low-rank adapters offer a parameter-efficient means to improve the performance of large language models (LLMs) across tasks. Hence, it is assumed that they can also be used to improve the support of low-resource languages in LLMs, which currently serve English far better than the thousands of other languages spoken worldwide. However, this assumption has never been verified empirically under a shared multilingual evaluation protocol. We address exactly this gap by comparing eight low-rank adaptation (LoRA) methods on a frozen base model, using 146 training runs across 200 unique languages and more than 10,000 evaluations. Our results suggest that a reduced-scale search can identify viable configurations of LoRA methods but does not reliably predict their relative performance, as the top-ranking configuration is never best at scale. LoRA reliably improves language-model fit on translation data, consistently degrades reading comprehension, and benefits low-resource languages most. These findings suggest that LoRA methods should be regarded as targeted tools rather than general-purpose means for improving the multilingual capabilities of LLMs.
| Comments: | To Appear - EACL 2027 (Findings) |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.23158 [cs.CL] |
| (or arXiv:2611.23158v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23158 arXiv-issued DOI via DataCite |
Submission history
From: Nikit Srivastava [view email]
[v1]
Tue, 17 Nov 2026 18:11:42 UTC (86 KB)
[v2]
Wed, 18 Nov 2026 08:38:14 UTC (89 KB)
来源:arXiv cs.CL · arxiv.org