arXiv cs.CL· Neeraja Kirtane, Kuan-Hao Huang·· 3 天前AI 评分26
语言引导实现多语言上下文学习：语言向量方法
Language Steering for Multilingual In-Context Learning
AI 导读
研究者提出"语言向量"方法，通过计算平行语料在特定层的平均激活差异，在推理时对模型隐藏状态施加偏移，无需更新参数即可提升多语言上下文学习效果。该方法在覆盖 19 种语言、3 个模型的 3 项任务上均优于英文少样本提示，且向量呈现可解释的语言结构——相近语言形成紧密聚类，跨任务和跨语言迁移效果更好。
正文 · 原文
Abstract:If multilingual language model representations contain separable language-associated structure, then shifting between languages may require only simple activation-space interventions. To test this hypothesis, we study multilingual in-context learning (ICL) as a controlled setting: few-shot demonstrations are provided in English while the test query is in a target language, creating a clear cross-lingual transfer challenge. We propose language vectors, which are computed as mean activation differences between parallel source and target language examples at a particular layer and applied as inference-time offsets to the model's hidden states. Across three tasks spanning 19 languages and three models, our method improves over English few-shot prompting without any parameter updates. Beyond performance, the vectors encode interpretable linguistic structure: closely related languages form tight clusters, and vectors transfer effectively across tasks and languages, with closely related languages' vectors recovering more of the gain than distant ones. Together, these findings suggest that language occupies structured directions in multilingual activation space, and that inference-time interventions on English demonstrations can narrow the cross-lingual ICL gap.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2602.02326 [cs.CL] |
| (or arXiv:2602.02326v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2602.02326 arXiv-issued DOI via DataCite |
Submission history
From: Neeraja Kirtane [view email]
[v1]
Mon, 2 Feb 2026 16:52:09 UTC (734 KB)
[v2]
Wed, 1 Apr 2026 17:54:08 UTC (727 KB)
[v3]
Wed, 14 Oct 2026 15:10:55 UTC (7,157 KB)
来源：arXiv cs.CL · arxiv.org