arXiv cs.CL· David Porte\v{s}, Ale\v{s} Hor\'ak·· 9 天前AI 评分25
Prosody-to-Text：从低通滤波语音预测文本
Prosody-to-Text: Predicting text from low-pass filtered speech
AI 导读
研究者提出"韵律到文本"任务，用仅保留 12 个最低 Mel 频段（约 450Hz 低通滤波）的语音微调 Whisper，从韵律模式中恢复原始句子，取得 WER 36% 的结果，10% 的语句被完全还原，40% 的语句 WER 不高于 25%。在给定正确前缀时，下一个 token 的预测准确率为 79%。结果表明低频语音特征与词汇内容的关联比此前认为的更强，或可用于引导现代 LLM 的文本生成。
正文 · 原文
Abstract:While predicting prosody from text is an established task in the field, the opposite direction, predicting text that fits a given prosodic pattern, remains largely overlooked. We find this unfortunate, because this opposite direction could lead to some very interesting use cases. Therefore, in this paper, we make the first steps in the prosody-to-text direction by inves- tigating how much of the original sentence can be recovered from its prosodic pattern. To this end, we fine-tune the Whis- per model using only the 12 lowest Mel bins (low-pass filter with approximately 450Hz cutoff), and obtain surprisingly accurate results (WER 36%), with 10% of utterances be- ing recovered perfectly, and 40% of utterances having Word Error Rate at or below 25%. We also find that, given the correct prefix, the next token was predicted correctly in 79% of cases. Our results suggest that the relationship between low-frequency speech features and lexical content is much stronger than previously thought, and we believe that direct- ing more attention to this topic might open the door to new applications, such as using prosody to guide text generation of modern LLMs
| Comments: | This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.11544 [cs.CL] |
| (or arXiv:2610.11544v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11544 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: David Porteš [view email]
[v1]
Thu, 8 Oct 2026 09:12:02 UTC (496 KB)
来源：arXiv cs.CL · arxiv.org