arXiv cs.CL· Ali Shendabadi, Parnia Izadirad, Mostafa Salehi·· 12 小时前AI 评分18
基于 Whisper 的波斯语语音情感识别：ASR 适配与表征降维研究
A Study of ASR Adaptation and Representation Dimensionality Reduction in Persian Speech Emotion Recognition Using Whisper
AI 导读
研究提出基于 Whisper 的波斯语语音情感识别框架，用 PCA 对 Whisper 编码器提取的帧级嵌入向量降维，省去可学习投影层并大幅减少可训练参数。在 ShEMO 数据集上的说话人无关评测显示，PCA 降维持续提升情感识别性能，同时降低训练延迟和内存占用。在波斯语 ASR 任务上微调 Whisper 对下游 SER 仅带来小幅提升，表明语言适配向情感表征的迁移有限。
正文 · 原文
Abstract:Speech Emotion Recognition (SER) in low-resource languages remains a challenging problem due to limited labeled data. In this work, we study the use of Whisper for Persian SER with a particular focus on representation dimensionality reduction and language-specific model adaptation. We propose a SER framework in which frame-level embeddings extracted from the Whisper encoder are reduced in dimensionality using PCA, eliminating the need for learned projection layers and substantially reducing the number of trainable parameters. The reduced representations are aggregated using an attention-based pooling mechanism and classified with a lightweight prediction head. In addition, we investigate whether fine-tuning Whisper on a Persian automatic speech recognition (ASR) task improves downstream SER performance. Experiments conducted on the ShEMO dataset under a speaker-independent evaluation protocol show that PCA-based dimensionality reduction consistently improves emotion recognition performance while reducing training latency and memory usage. ASR fine-tuning yields only modest gains for SER, suggesting limited transfer from language adaptation to emotion-related representations under the evaluated conditions. These findings provide practical insights into the efficient use of large pretrained speech models for emotion recognition in low-resource languages.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Sound (cs.SD) |
| Cite as: | arXiv:2608.05165 [cs.CL] |
| (or arXiv:2608.05165v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.05165 arXiv-issued DOI via DataCite |
Submission history
From: Ali Shendabadi [view email]
[v1]
Tue, 26 May 2026 13:23:02 UTC (129 KB)
[v2]
Fri, 7 Aug 2026 07:14:35 UTC (61 KB)
[v3]
Mon, 2 Nov 2026 17:29:11 UTC (71 KB)
来源：arXiv cs.CL · arxiv.org