UXBench Pro：多轮对话交互中的个性化用户体验基准测试
UXBench Pro: Benchmarking Personalized User Experience in Multi-Turn Dialogue Interactions
研究者推出 UXBench Pro，包含来自 12 个任务场景、82 个领域的 1000 个真实用户交互测试实例，每个实例配有刻画七种可解释行为特征的 FACTORS 用户画像。该基准提出双视角评估范式，结合个性化 User Reward Model（URM）与用户模拟器 Sim4Eval，并引入 URMBench 和 USimBench 两个元基准检验评估器还原真实人类偏好与行为的可靠性。
Authors:Mengze Hong, Zeyang Lei, Wenbo Shang, Xia Zeng, Xiying Zhao, Qi Zhu, Chen Jason Zhang, Di Jiang, Taiming Fu, Qiongyi Zhou, Qinghe Chang, Fubao Zhang, Chenxuan Ma, Minlong Peng, Jinfeng Huang, Zineng Zhou, Jindou Wu, Muge Qi, Sijun He, Xin Cui, Di Liang, Yuan Hua, Davey Chen
Abstract:Evaluating user experience (UX) with automated computational methods has gained increasing attention, supported by empirical evidence from UXBench. However, binary preference prediction provides limited insight, while relying on a single user-agnostic reward model overlooks the inherent heterogeneity of users, whose expectations can differ substantially. In this paper, we present UXBench Pro, comprising 1{,}000 test instances derived from real user interactions across 12 task scenarios and 82 domains. Each instance is paired with a FACTORS user profile that characterizes the user through seven interpretable behavioral facets, differentiating user groups. To provide richer evaluation insights, we introduce a dual-perspective paradigm that combines a personalized User Reward Model (URM) for third-person judgment with Sim4Eval, a user simulator that enables multi-turn interactions and provides first-person evaluation across four cognitive state dimensions. To assess the reliability of these based evaluators, we further introduce two meta-benchmarks, URMBench and USimBench, that evaluate how faithfully they reproduce real human preferences and behaviors. Extensive experiments reveal seven key findings that highlight the importance of user modeling and multi-perspective evaluation, offering a fresh perspective on user-centric benchmarking and motivating personalized model optimization.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.11638 [cs.CL] |
| (or arXiv:2610.11638v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11638 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mengze Hong [view email]
[v1]
Thu, 8 Oct 2026 10:15:55 UTC (11,665 KB)
来源：arXiv cs.CL · arxiv.org