arXiv cs.CL· Moran Yanuka, Raja Giryes, Moris Alper·· 9 天前AI 评分24
多语言语音模型中的音系干扰问题研究
Phonological Interference in Multilingual Speech Models
AI 导读
研究者发现音素级语音模型存在系统性失败模式——音系干扰：模型默认输入为单一语言并强加其音系规则，覆盖与之冲突的局部音素决策。在语码转换输入上，两个语音转音素识别器和一个音素条件TTS模型丢失了32%至79%的跨语言特有音素。研究提出窗口化语言估计（WLE）推理时修复方法，在三个模型上消除了34%至69%的干扰，且识别器的单语性能基本不变。
正文 · 原文
Abstract:Phoneme-level models transcribe or generate speech as a sequence of phonemes, the smallest sound units that distinguish words. These models enable fine-grained pronunciation control and understanding, yet often fail on input that does not match any single training language, such as speech alternating between two languages, known as code-switching, or low-resource languages absent from training. We identify a systematic failure mode behind this, phonological interference: models assume the input is in a single language and impose its phonology, overriding local phoneme-level decisions that conflict with the assumed language. We measure interference by how often a model retains phonemes that one language has but the other lacks. On code-switched input, two phone recognizers (speech-to-phoneme models) and a phoneme-conditioned text-to-speech model lose 32% to 79% of these phonemes, but lose far fewer of the phonemes both languages share. On unseen languages, we find that phone recognizers impose the phonology of the training language they assign to the speech, and the more confident the assignment, the more they lose phonemes the unseen language has but the assigned language lacks. We probe the models' language estimate from their internal activations, and trace interference to a low dimensional subspace. On monolingual speech, steering this subspace toward another language makes the model lose the phonemes that only the original language uses and produce phonemes that only the target language has. We introduce windowed language estimation (WLE), an inference time repair that replaces the model's language estimate in this subspace with one computed from a short window around each position. On code-switched input, WLE removes 34% to 69% of the interference in all three models, and in the recognizers it leaves monolingual performance essentially unchanged.
| Comments: | Preprint |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.11275 [cs.CL] |
| (or arXiv:2610.11275v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11275 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Moran Yanuka [view email]
[v1]
Thu, 8 Oct 2026 05:38:01 UTC (272 KB)
来源：arXiv cs.CL · arxiv.org