arXiv cs.CL· Shaanil Punglia, Dadhichi Telwadkar, Vaibhav Rathore, Siddharth Sureka, Neha Bhargava·· 11 小时前AI 评分22
语言子空间干预能否提升多语言推理？研究揭示准确率与语言保真度的权衡
Beyond Accuracy: What Language-Subspace Interventions Capture, Remove, and Improve
AI 导读
研究在 MATH 和 MGSM 上评估四种开源权重 LLM 的语言子空间干预：Mean-SVD 与 DiffPCA 构建的子空间虽能捕捉大量翻译相关变化，但投影后线性探针仍几乎完美识别语言，说明二者均未穷尽语言信息。
正文 · 原文
Abstract:Language-subspace interventions aim to improve multilingual reasoning by suppressing language-related components in mid-layer activations. Using MATH and MGSM across four open-weight LLMs, we ask three questions: do estimated language subspaces capture translation-associated variation? Does projecting them out remove language information? And do they improve accuracy without compromising response language? For the first question, we build subspaces using the existing Mean-SVD construction and our DiffPCA. Both capture substantial translation-associated variation and show similar patterns across model depth. For the second, we project out each subspace and train linear probes to classify language from the projected activations. The probes still identify language almost perfectly, showing that neither subspace exhaustively captures language information. For the third, we evaluate several intervention configurations on Qwen2.5-7B, by varying subspace creation methods and intervention operators. We find that the intervention that most improves MGSM accuracy sacrifices response-language fidelity, with only one intervention improving joint accuracy$\unicode{x2014}$answers that are correct and in the target language$\unicode{x2014}$slightly, by 1.60 points. Interestingly, subtle changes in the prompt alone can reproduce the accuracy gains albeit with significant damage to language fidelity. We argue that causal interventions must be judged on more than accuracy, to fully account for language substitution trade-offs.
| Comments: | AACL-IJCNLP 2026 Workshop Multi-LLL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.05278 [cs.CL] |
| (or arXiv:2611.05278v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05278 arXiv-issued DOI via DataCite |
Submission history
From: Shaanil Punglia [view email]
[v1]
Tue, 3 Nov 2026 17:58:52 UTC (241 KB)
[v2]
Fri, 6 Nov 2026 04:35:03 UTC (241 KB)
来源：arXiv cs.CL · arxiv.org