arXiv cs.CL· Andrey Goncharov, Nikolai Kondusov, Alexey Zaytsev·· 12 小时前AI 评分31
在潜空间中引导语言以缓解非预期语码转换
Language steering in latent space to mitigate unintended code-switching
AI 导读
研究提出一种轻量级推理时方法“潜空间语言引导”，通过对平行翻译做 PCA 识别语言方向，并沿这些轴引导 token 嵌入以控制语言身份。该方法将下一 token 分布散度最多降低 55%，Code-Switching Index 降低 63–99%，且计算开销可忽略、仅需少量平行数据校准。单个主成分即可实现 95–99% 的语言分类准确率，论文已被 AINL 2026 接收。
正文 · 原文
Abstract:Multilingual Large Language Models (LLMs) often exhibit hallucinations such as unintended code-switching, reducing reliability in downstream tasks. We propose latent-space language steering, a lightweight inference-time method that identifies language directions via Principal Component Analysis (PCA) on parallel translations and steers token embeddings along these axes to control language identity. Our approach mitigates code-switching (reduced next-token distributional divergence by up to 55\%, reduced Code-Switching Index by 63--99\%) while preserving broad semantic direction with negligible computational overhead and requires only minimal parallel data for calibration. Additionally, we evaluate language classification using a single principal component (95-99\% accuracy) and further analyze the layer-wise evolution of language representations.
| Comments: | Published at AINL 2026 |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2510.13849 [cs.CL] |
| (or arXiv:2510.13849v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2510.13849 arXiv-issued DOI via DataCite |
Submission history
From: Andrey Goncharov [view email]
[v1]
Sat, 11 Oct 2025 19:49:38 UTC (2,821 KB)
[v2]
Mon, 2 Mar 2026 12:36:21 UTC (2,810 KB)
[v3]
Wed, 15 Apr 2026 11:13:28 UTC (2,807 KB)
[v4]
Fri, 16 Oct 2026 17:02:53 UTC (2,820 KB)
来源：arXiv cs.CL · arxiv.org