arXiv cs.AI· Jiaqi Qiao, Yifan Lyu, Xiujuan Xu·· 12 小时前AI 评分22
Transformer-GAT：融合 Transformer 与图注意力网络的对话情绪识别框架
Cross-Modal Emotion Understanding: A Transformer-GAT Approach for Dialogue Emotion Recognition
AI 导读
研究者提出 Transformer-GAT 混合框架，用 Transformer 捕捉全局语义信息、用图注意力网络建模模态间细粒度关系，实现跨模态情绪理解。在 IEMOCAP 和 MELD 数据集上，该模型加权 F1 分别达到 72.45% 和 77.37%，优于现有 SOTA 方法。
正文 · 原文
Abstract:Multimodal emotion recognition is a key research area in affective computing, with applications in sentiment analysis, intelligent customer service, and human-computer interaction. However, existing methods often rely on single-modal features or simple multimodal fusion, failing to capture the synergy between global and local contexts, which limits model performance and emotion understanding. To address this challenge, we propose Transformer-GAT, a hybrid framework that combines Transformer and the Graph Attention Network to enable cross-modal emotion understanding. The Transformer is used to capture global semantic information, while the Graph Attention Network is employed to model fine-grained relationships between modalities, thereby enhancing the representation of emotional features. Experiments on the IEMOCAP and MELD datasets show that our model achieves weighted F1 scores of 72.45% and 77.37%, outperforming state-of-the-art methods. These results demonstrate that Transformer-GAT effectively integrates multimodal features, balances global and local contexts, and provides deeper emotional insights, offering new directions for multimodal emotion computing.
| Comments: | 12 pages, 9 figures |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.29556 [cs.AI] |
| (or arXiv:2609.29556v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.29556 arXiv-issued DOI via DataCite |
Submission history
From: Yifan Lyu [view email]
[v1]
Wed, 2 Sep 2026 07:50:49 UTC (3,574 KB)
[v2]
Fri, 16 Oct 2026 02:40:17 UTC (3,574 KB)
来源：arXiv cs.AI · arxiv.org