跳到正文
arXiv cs.CL· Nikit Srivastava, Joel Dag, Ren\'e Speck, Axel-Cyrille Ngonga Ngomo·· 23 小时前AI 评分34

MoE-LoRA 方法用于多语言适配:一项系统性实证研究

MoE--LoRA Methods for Multilingual Adaptation: A Systematic Empirical Study

AI 导读

一项系统性实证研究在冻结基座模型上对比了 8 种 LoRA 方法,覆盖 200 种语言、146 次训练运行和超过 10,000 次评测。结果显示 LoRA 能稳定提升翻译数据的语言模型拟合度、持续降低阅读理解表现,且对低资源语言收益最大;小规模搜索可找到可行配置,但无法可靠预测其相对性能。研究据此认为 LoRA 应被视为有针对性的工具,而非提升 LLM 多语言能力的通用手段。

正文 · 原文

View PDF HTML (experimental)

Abstract:Routed low-rank adapters offer a parameter-efficient means to improve the performance of large language models (LLMs) across tasks. Hence, it is assumed that they can also be used to improve the support of low-resource languages in LLMs, which currently serve English far better than the thousands of other languages spoken worldwide. However, this assumption has never been verified empirically under a shared multilingual evaluation protocol. We address exactly this gap by comparing eight low-rank adaptation (LoRA) methods on a frozen base model, using 146 training runs across 200 unique languages and more than 10,000 evaluations. Our results suggest that a reduced-scale search can identify viable configurations of LoRA methods but does not reliably predict their relative performance, as the top-ranking configuration is never best at scale. LoRA reliably improves language-model fit on translation data, consistently degrades reading comprehension, and benefits low-resource languages most. These findings suggest that LoRA methods should be regarded as targeted tools rather than general-purpose means for improving the multilingual capabilities of LLMs.
Comments: To Appear - EACL 2027 (Findings)
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2611.23158 [cs.CL]
  (or arXiv:2611.23158v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.23158

arXiv-issued DOI via DataCite

Submission history

From: Nikit Srivastava [view email]
[v1] Tue, 17 Nov 2026 18:11:42 UTC (86 KB)
[v2] Wed, 18 Nov 2026 08:38:14 UTC (89 KB)

来源:arXiv cs.CL · arxiv.org