UXBench：首个基于真实用户反馈的 AI 助手用户体验基准发布
UXBench: Benchmarking User Experience in AI Assistants
UXBench 是首个基于真实用户反馈信号构建的用户体验基准，包含 UX Judge、UX Eval、UX Recovery 三项任务，测试集 7,400 条，取自某主流中文 AI 助手超 70K 条交互日志，覆盖 8 个场景、83 个领域。研究对 26 个前沿语言模型进行实验，记录六项发现，表明用户反馈预测是可学习的能力。该基准已被 EMNLP 2026 主会接收，项目已完整发布。
Authors:Mengze Hong, Xia Zeng, Zeyang Lei, Sheng Wang, Chen Jason Zhang, Di Jiang, Taiming Fu, Jinfeng Huang, Mengqiao Liu, Qinghe Chang, Haosheng Zou, Qiongyi Zhou, Sijun He, Simonjmdeng, Minlong Peng, Di Liang
Abstract:As AI assistants serve millions of users daily, evaluating user experience (UX) beyond general model capability has become increasingly important. We present \textbf{UXBench}, the first user-centric benchmark grounded in real user feedback signals for evaluating preference alignment and dialogue generation. The benchmark consists of three interconnected tasks, UX Judge, UX Eval, and UX Recovery, with 7,400 test instances extracted from over 70K interaction logs of a mainstream Chinese AI assistant. The dataset closely reflects real user distributions, covering 8 scenarios, 83 domains, and diverse failure patterns that pose severe challenges. Extensive experiments on 26 frontier language models provide novel insights into how well models perceive user experience and how improvements in model capability contribute to better dialogue engagement. Through analyses of model behavior and performance gaps, we document six important findings, demonstrating that user feedback prediction is a learnable capability and revealing different aspects that influence user experience. UXBench establishes a new evaluation landscape and calls for greater attention to tailored UX optimization, contributing toward a user-centric scaling law for the development of successful AI assistants. The full project is released at this https URL.
| Comments: | Accepted by EMNLP 2026 Main Conference |
| Subjects: | Computation and Language (cs.CL); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2606.09570 [cs.CL] |
| (or arXiv:2606.09570v5 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.09570 arXiv-issued DOI via DataCite |
Submission history
From: Mengze Hong [view email]
[v1]
Mon, 8 Jun 2026 14:44:01 UTC (7,752 KB)
[v2]
Tue, 9 Jun 2026 13:55:26 UTC (7,751 KB)
[v3]
Tue, 14 Jul 2026 09:11:36 UTC (7,752 KB)
[v4]
Sun, 27 Sep 2026 10:01:27 UTC (7,295 KB)
[v5]
Mon, 2 Nov 2026 07:17:36 UTC (7,295 KB)
来源：arXiv cs.CL · arxiv.org