arXiv cs.CL· Tobias Hallmen, Fabian Deuser, Robin-Nico Kampa, Norbert Oswald, Elisabeth Andr\'e·· 12 小时前AI 评分50
解析文本低估了视觉语言模型：细粒度面部情绪识别可从低于专家一致到高于专家一致
Parsed Text Understates Vision-Language Models: From Below to Above Expert Agreement on Fine-Grained Facial Emotion
AI 导读
研究指出 EmoNet-Face-HQ 基准从生成文本读取答案、要求每张脸固定情绪数量的协议本身会决定结果。改为对 40 个情绪类别逐类做 yes/no 提问并按“yes”概率打分后，11 个开放权重 VLM 在专家最一致的 5 个类别上平均二次加权 κ 从 0.425 升至 0.553，全部超过两位专家之间 0.468 的一致水平。
正文 · 原文
Abstract:EmoNet-Face-HQ is a benchmark of 2,500 generated portraits, each rated by four psychology experts on 40 emotion categories. It concludes that vision-language models (VLMs) on average offer no dependable improvement over random guessing and that a fine-tuned model is required. The benchmark reads each model's answer from generated text, and its zero-shot prompt demands a fixed number of emotions per face. Its own results already show that this protocol can decide the outcome: one VLM moves from chance to close to expert agreement when only the prompt changes. We keep the images, expert ratings and metric and change only how the answer is asked for and read: one yes/no question per category, scored by the probability of "yes". Scored from parsed text on the five categories experts agree on most, eleven open-weight VLMs average a quadratic-weighted $\kappa$ of 0.425 against individual experts, below the 0.468 at which two experts agree. Scored from yes-probabilities, the same models average 0.553, and all eleven agree with an individual expert better than a second expert does. The VLMs' mean is level with the benchmark's fine-tuned model (0.555), and three of them significantly exceed it without any training. Only one exceeds a stricter reference, the mean of three experts scored against the fourth (0.563). Verdicts that VLMs fail at graded facial affect ratings may therefore describe the readout more than the model. A fine-tuned model should have to beat a pretrained VLM read this way before it is called necessary, and the benchmark's own does not clearly do so.
| Comments: | Preprint. 10 pages, 2 figures; 10-page supplementary material as ancillary file. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.08162 [cs.CV] |
| (or arXiv:2610.08162v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.08162 arXiv-issued DOI via DataCite |
Submission history
From: Tobias Hallmen [view email]
[v1]
Tue, 6 Oct 2026 11:14:23 UTC (106 KB)
[v2]
Sat, 31 Oct 2026 10:08:27 UTC (488 KB)
来源：arXiv cs.CL · arxiv.org