arXiv cs.CL· Jie Chi, Zakaria Aldeneh, Masha Fedzechkina, Maureen de Seyssel, Natalie Schluter·· 2 天前AI 评分20
利用语音基础模型的语言选择性维度进行语码转换检测
Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models
AI 导读
研究者提出一种直接从语音中检测语码转换（CS）的轻量方法，利用预训练语音编码器帧级的语言选择性隐藏状态维度，聚合激活值来预测一句话中是否同时出现两种语言。在 CS-FLEURS 的 14 个语言对上，Whisper-small 编码器取得平均 F1 0.798，高于 CNN 的 0.776 和 HuBERT 线性基线的 0.751。
正文 · 原文
Abstract:Code-switching (CS), the alternation between languages within an utterance, is common in multilingual speech but underrepresented in pretraining corpora. Increasing CS coverage requires finding relevant utterances in large speech collections so that they can be included in training data. One way to find such utterances is to transcribe the speech and detect language alternation in the text, but this depends on obtaining accurate multilingual transcriptions. We propose a lightweight method that detects CS directly from speech. It uses frame-level language information in pretrained speech encoders to determine whether both languages are present within an utterance. Specifically, we identify language-selective hidden-state dimensions using monolingual recordings and aggregate their frame-level activations into a CS prediction. With a small CS set used to tune a decision rule, we show that the Whisper-small encoder achieves a mean F1 of $0.798$ across 14 language pairs in CS-FLEURS, compared with $0.776$ for a CNN and $0.751$ for a HuBERT linear baseline trained for monolingual language identification. Our findings suggest that pretrained speech encoders already encode the information needed to identify CS utterances, and they provide a lightweight tool for building more representative multilingual corpora.
| Comments: | submitted to ICASSP 2027 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.18726 [cs.CL] |
| (or arXiv:2610.18726v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18726 arXiv-issued DOI via DataCite |
Submission history
From: Jie Chi [view email]
[v1]
Tue, 13 Oct 2026 17:05:22 UTC (24 KB)
[v2]
Wed, 14 Oct 2026 22:26:33 UTC (23 KB)
来源：arXiv cs.CL · arxiv.org