arXiv cs.CL· Shin Yamaguchi, Masaya Taniguchi, Tatsuki Kuribayashi, Taro Watanabe, Yoshinobu Kano·· 13 小时前AI 评分24
语言模型能跨语言迁移层级泛化能力吗？EMNLP 2026 研究给出条件性答案
Once a Tree, Always a Tree? Cross-lingual Transfer of Hierarchical Generalization in Language Models
AI 导读
研究提出跨语言贫困刺激（CPOS）实验设计，检验语言模型能否将源语言（L1）中的层级结构证据迁移到新语言（L2）。结果显示模型只能部分迁移，仅产生微弱但可检测的层级偏好，且迁移强度因语言对而异。进一步消融发现词汇重叠对层级句法泛化的跨语言迁移至关重要，说明这种迁移只是有条件成立。
正文 · 原文
Abstract:Language models (LMs) often exhibit successful cross-lingual transfer through continued (pre)training, still their underlying process remains unclear. One possibility is that LMs can leverage the evidence toward language-universal principles, namely, hierarchical structure, from a source language (L1) to a new language (L2), while many other non-linguistic explanations, e.g., improved initialization of position embeddings, can be potentially possible. In this study, to quantify the cross-linguistic transferability of linguistic evidence in cross-lingual LM training, we introduce a new experimental design, namely, a cross-lingual poverty-of-stimulus (CPOS) setting that ablates how LMs' hierarchical generalization in L2 can be led by structural evidence provided in L1. We contrast counterfactual L1 and L2 learning scenarios where the L2 corpus is ambiguous between linear and hierarchical grammar rules, while the amount of positive evidence for the hierarchical rule is controlled on the L1 side. Our experimental results show that LMs can partially, yet not perfectly, transfer L1 structural evidence to L2, yielding only a small but detectable hierarchical bias, while the strength of transfer depends on language pairs. We further ablate the conditions for successful transfer, motivated by recent reports on the substantial impact of vocabulary overlap. We indeed find that vocabulary overlap is crucial even for the cross-lingual transfer of hierarchical syntactic generalization. This suggests that this also explains the language-pair-dependent tendencies in the main experiments and suggests that the transfer of linguistic evidence in continued new-language learning is only conditionally achievable.
| Comments: | EMNLP 2026 main |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.30960 [cs.CL] |
| (or arXiv:2610.30960v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30960 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shin Yamaguchi [view email]
[v1]
Thu, 22 Oct 2026 09:07:44 UTC (742 KB)
来源：arXiv cs.CL · arxiv.org