arXiv cs.CL· Yue Wu, Chengbing Wang, Yimeng Bai, Xiaoyan Zhao, Yang Zhang, Fuli Feng·· 10 小时前AI 评分22
PrefReward：用用户偏好矩阵实现个性化文本生成
PrefReward: Learning User Preference Matrix for Personalized Text Generation
AI 导读
PrefReward 是一个偏好感知生成框架，通过结构化偏好矩阵显式建模用户风格，并将其作为奖励信号接入解码过程。该框架分两阶段：先提取用户专属偏好矩阵以总结个人风格倾向，再用基于 KL 散度的奖励函数引导生成。在 LongLaMP 数据集上，PrefReward 在生成质量和个性化可解释性上均优于检索类基线与上下文引导方法。
正文 · 原文
Abstract:Large Language Models (LLMs) have demonstrated remarkable ability in generating personalized content by leveraging user histories and contextual cues. However, most existing personalization approaches rely on implicit representations within model parameters, making it difficult to interpret user-specific preferences or effectively handle long-context dependencies. To address these challenges, we propose PrefReward, a novel preference-aware generative framework that explicitly models user styles through a structured preference matrix and integrates it into the decoding process as a reward signal. PrefReward consists of two stages: (1) extracting a user-specific preference matrix that summarizes individual stylistic tendencies, and (2) using the matrix to guide generation via a KL-divergence-based reward function. Experiments on the LongLaMP dataset show that PrefReward outperforms retrieval-based baselines and context steering method in both generation quality and personalization interpretability.
| Comments: | added several analyses and visualizations, typos corrected |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.21067 [cs.CL] |
| (or arXiv:2607.21067v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.21067 arXiv-issued DOI via DataCite |
Submission history
From: Yue Wu [view email]
[v1]
Thu, 23 Jul 2026 09:00:20 UTC (1,810 KB)
[v2]
Tue, 10 Nov 2026 07:43:54 UTC (417 KB)
来源：arXiv cs.CL · arxiv.org