arXiv cs.CL· Yijiang River Dong, Zexue He, Cheng Yu, Tiancheng Hu, Michael J Ryan, Jan Batzner, Zheng Hui, Shaolong Wu, Orestis Papakyriakopoulos, Alex Pentland, Nigel Collier, Jiaxin Pei·· 13 小时前AI 评分47
Memory-Aware Reward Modeling：让 LLM 记忆使用对齐人类偏好
Memory-Aware Reward Modeling: Learning Human Preferences for LLM Memory Use
AI 导读
研究者收集 87 名参与者基于自身 ChatGPT 查询与个人记忆的约 1,000 组成对判断，发现带记忆的回复并不总被偏好，自动评估器与用户一致性有限且明显更偏爱记忆条件回复。为此提出记忆感知奖励模型，偏好预测准确率达 72.8%，超过现有基线。
正文 · 原文
Authors:Yijiang River Dong, Zexue He, Cheng Yu, Tiancheng Hu, Michael J Ryan, Jan Batzner, Zheng Hui, Shaolong Wu, Orestis Papakyriakopoulos, Alex Pentland, Nigel Collier, Jiaxin Pei
Abstract:Memory enables language model assistants to maintain continuity across sessions and generate personalized responses, but knowing more about a user does not necessarily produce responses they prefer. Existing evaluations assess memory capabilities without directly establishing whether memory use benefits the user. To study this gap, we collect approximately 1,000 pairwise judgments from 87 participants using their own ChatGPT queries and personal memories. We find that memory-conditioned responses are not consistently preferred over responses generated without memory and in some cases, users strongly prefer the latter. Automatic evaluators also show limited agreement with users and favor memory-conditioned responses substantially more often than users do. Motivated by these gaps, we introduce a memory-aware reward model that learns from user judgments, achieving 72.8\% preference prediction accuracy and outperforming the existing baselines. Our findings motivate evaluation and optimization objectives that align memory use with the preferences of the people whose information is being used.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.31272 [cs.CL] |
| (or arXiv:2610.31272v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31272 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yijiang Dong [view email]
[v1]
Thu, 22 Oct 2026 13:43:33 UTC (440 KB)
来源：arXiv cs.CL · arxiv.org