arXiv cs.CL· Atai Waris Khan, Abdullah Mazhar, Raghu Kumar, Md Shad Akhtar·· 2 天前AI 评分24
TITAN：面向低资源多语言翻译的翻译电路发现与定向优化
Translation Circuit Discovery and Targeted Optimization for Low-Resource Multilingual Translation
AI 导读
研究者提出 TITAN 机制框架，结合因果干预、稀疏特征分解与定向偏好优化，识别并优化 Transformer 中关键的翻译电路，仅更新不到 5% 的模型参数。
正文 · 原文
Abstract:Recent multilingual large language models achieve strong translation performance, yet the internal mechanisms responsible for low-resource multilingual translation remain insufficiently understood. In this paper, we study translation circuits in multilingual decoder-only LLMs for Hindi-to-Urdu and Hindi-to-Sindhi (Devanagari). We propose TITAN, a mechanistic framework that combines causal intervention, sparse feature decomposition, and targeted preference optimization to identify and optimize crucial translation components in the transformer. Unlike conventional full-parameter fine-tuning, TITAN restricts optimization to sparse, causally important circuits, enabling scalable and parameter-efficient adaptation while updating fewer than 5% of model parameters. Experimental results show that TITAN achieves BLEU improvements of 33.75% on Hindi-to-Urdu and 45.86% on Hindi-to-Sindhi over the strongest multilingual baselines. Human expert evaluation further confirms substantial gains in adequacy and fluency across both language pairs. Additional analyses reveal that multilingual translation is implemented through sparse, structured circuits operating over a shared semantic subspace rather than through an English-centric pivot representation. Overall, our findings demonstrate that mechanistic circuit discovery can directly improve low-resource multilingual translation while providing interpretable insights into multilingual transformer behavior.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.21456 [cs.CL] |
| (or arXiv:2610.21456v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21456 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Abdullah Mazhar [view email]
[v1]
Thu, 15 Oct 2026 07:57:10 UTC (2,996 KB)
来源：arXiv cs.CL · arxiv.org