arXiv cs.CL· Zihan Chen, Lei Nico Zheng, Di Zhu·· 13 小时前AI 评分31
研究揭示：用户声称的偏好与 AI 回复实际奖励的特征关联有限
Abstract Priorities, Concrete Rewards: How Stated Preferences Relate to What People Reward in AI Responses
AI 导读
基于 PRISM 数据集对 1,396 名参与者、21 个 LLM 的 29,347 条回复评分分析发现，回复长度、格式和拒答措辞等调查未问及的特征，与评分的关联至少与所调查属性的文本代理指标一样强。参与者若在书面反馈中要求更短回答，评分时对长度的权重更低，反之亦然；但其所声明的属性优先级无法预测这一权重。利用个人信息预测留出选择的最大提升仅 0.13 个百分点，而更强的池化模型为 1.11。
正文 · 原文
Abstract:AI assistants built on large language models (LLMs) are trained on human feedback and increasingly personalized to users' stated preferences. Yet how closely these preferences reflect what people reward in responses remains unclear. We compare stated and revealed preferences in the public PRISM dataset, in which 1,396 participants reported how much seven response attributes mattered to them and then rated 29,347 responses from 21 LLMs. Among responses to the same prompt, features the survey never asked about, such as response length, formatting and refusal language, were at least as strongly associated with ratings as text proxies of the surveyed attributes. Whether stated priorities matched ratings depended on how attributes were measured: participants who prioritized an attribute favoured responses from models that others perceived as strong on it, but not responses that scored higher on text proxies for it. Written feedback was more telling: participants who asked for shorter answers, even in earlier conversations, gave length less weight in their ratings, and those asking for longer answers gave it more. Their relative priorities over the surveyed attributes, however, did not predict this weight. Reasons given for choices reflected perceived attributes more than measured features. Participants also differed stably in how much they weighted several concrete features, especially length, but with a median of 6 conversations per participant, these weights were estimated poorly. The largest gain from personal information in predicting held-out choices was 0.13 percentage points, compared with 1.11 for a stronger pooled model. Stated priorities thus track perceived attributes of AI responses but say little about the concrete features that ratings reward, a distinction that matters for preference elicitation and AI personalization.
| Comments: | 27 pages, 5 figures, 30 tables. Code and data are available from the authors upon request |
| Subjects: | Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| ACM classes: | H.1.2; H.5.2; I.2.7 |
| Cite as: | arXiv:2610.30763 [cs.HC] |
| (or arXiv:2610.30763v1 [cs.HC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30763 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zihan Chen [view email]
[v1]
Thu, 22 Oct 2026 05:34:33 UTC (111 KB)
来源：arXiv cs.CL · arxiv.org