arXiv cs.CL· Emily \"Ohman, Anna Koufakou·· 6 天前AI 评分22
同名情绪标签未必等价：GoEmotions、XED 与 BRIGHTER 数据集间的情绪标签差异研究
Fear Is Not Always Fear: Variation in Emotion Label Equivalence Across Datasets
AI 导读
研究测试了 GoEmotions、XED 和 BRIGHTER 共享的五个情绪标签，发现同一批文本上，一个 fear 分类器将 71% 标为恐惧，另一个仅 8%，但两者对文本排序基本一致。
正文 · 原文
Abstract:Emotion classifiers are routinely applied beyond their training data, on the assumption that same-named labels such as fear or surprise mean the same thing across resources. We test this assumption for five labels shared by GoEmotions, XED, and BRIGHTER, comparing models on identical inputs and under cross-corpus transfer. On the same texts, one fear classifier flags 71% as fearful and another 8%, yet the two largely agree on how the texts rank. Across 30 resource-label comparisons, training-label prevalence explains about half of this kind of scale mismatch, and a fitted intercept removes most of it. What recalibration cannot repair is larger: transferred models lose 55% of native discrimination in aggregate, agreement in ranking between classifiers is only weakly related to that loss, and for surprise the resources disagree about which texts express the emotion. The choice of training resource can therefore change conclusions: on two external tweet collections, BRIGHTER- and XED-trained classifiers give opposite answers about which shows more surprise, even after rank normalization, with intervals excluding zero in both directions. A shared emotion label name does not establish an interchangeable measurement target.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.12841 [cs.CL] |
| (or arXiv:2610.12841v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12841 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Emily Ohman [view email]
[v1]
Fri, 9 Oct 2026 00:55:54 UTC (476 KB)
来源：arXiv cs.CL · arxiv.org