arXiv cs.CL· Yuka Saito, Ikumi Numaya, Jun Suzuki, Reina Akama·· 10 小时前AI 评分31
研究：从对话文本中检测人机交互中的用户不满
Beyond Fluent Responses: Finding User Dissatisfaction in Human-AI Interactions
AI 导读
一项被 LUHME 2026 接收的研究分析了带自我报告不满标签的用户话语，检验能否从对话文本中检测用户不满。在英文交互中，使用用户话语的模型在 AUPRC 上比 17.8% 的不满标签基线最多高出 15 个百分点，说明用户话语中隐含不满线索。相对前一条用户话语的变化可提升检测效果，异常检测方法尤为有效。
正文 · 原文
Abstract:As large language models generate increasingly high-quality responses, recent work on response evaluation has moved beyond response-centered criteria, such as correctness and coherence, toward user-centered criteria, including user preferences and perceived usefulness. This shift is fundamentally important for developing technologies that are truly useful to real-world users. While conventional response evaluation quantifies the quality of the response itself, users' satisfaction in real conversations also depends on their subjective preferences and perceptions of usefulness, which do not necessarily align with response quality. This study investigates whether such user-centered signals can be detected from conversational text. Specifically, we analyze user utterances with self-reported dissatisfaction labels to examine whether user dissatisfaction in text can be detected. In English interaction, models using user utterances exceeded the 17.8% dissatisfaction label prevalence in AUPRC by up to 15 percentage points. These results suggest that user utterances reveal cues to implicit dissatisfaction. Furthermore, changes from the immediately preceding user utterance may improve dissatisfaction detection, with anomaly detection methods appearing particularly effective.
| Comments: | Accepted to LUHME 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.13894 [cs.CL] |
| (or arXiv:2611.13894v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13894 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuka Saito [view email]
[v1]
Tue, 10 Nov 2026 10:54:11 UTC (40,664 KB)
来源：arXiv cs.CL · arxiv.org