arXiv cs.AI· Yiquan Zhou, Yixuan Wang, Xiaozhe Qi, Xin Gao, Shihao Li·· 13 小时前AI 评分34
音色藏在哪里？语音信号中说话人信息的定位研究
Where Does Timbre Hide? Localizing Speaker Information in Speech Signals
AI 导读
研究以说话人身份为可测量代理，系统探测了语音中音色信息的分布位置：1–8 kHz 幅度与声道谱包络结构主导听感，8 kHz 以上频段仍具机器判别力，浅层 SSL 层暴露紧凑的说话人信息，token 泄漏程度取决于 tokenizer 与层选择。
正文 · 原文
Abstract:Timbre similarity is a key evaluation target for generative speech systems, including voice conversion, speaker-cloning text-to-speech, and zero-shot speech models. When the generated voice does not match the target speaker, the problem often reflects imperfect separation of speaker-related timbre from content, making it important to identify where timbre information resides and which representations still carry it. Using speaker identity as a measurable proxy, we probe frequency bands, phase/magnitude structure, WORLD source/filter decomposition, voiced/unvoiced regions, self-supervised learning (SSL) layers, EnCodec/HuBERT tokens, and a 45-listener test across Mandarin, English, and Japanese speech/singing data. Speaker information is distributed: 1--8 kHz magnitude and vocal-tract spectral-envelope structure dominate perception, above-8 kHz bands remain machine-discriminative, shallow SSL layers expose compact speaker information, and token leakage depends on tokenizer/layer choice. These findings indicate where generation systems should condition on or transform speaker information, and where content, codec, and token pipelines should audit residual speaker leakage.
| Comments: | accepted by SLT2026 |
| Subjects: | Sound (cs.SD); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.29150 [cs.SD] |
| (or arXiv:2610.29150v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29150 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yiquan Zhou [view email]
[v1]
Wed, 21 Oct 2026 03:35:42 UTC (953 KB)
来源：arXiv cs.AI · arxiv.org