arXiv cs.CL· Chaeho Lee, Junyup Kim, Wansoo Kim, Young Min Jung, Sang Ho Lee·· 10 小时前AI 评分29
利用心理物理注意分布实现眼动增强的奖励建模
Leveraging Psychophysical Attentional Distribution for Gaze-Augmented Reward Modeling
AI 导读
研究将人类字母识别中的注意分布引入奖励模型训练，把视觉注意建模为以当前注视点为中心的空间渐变分布，替代以往以注视点为中心的建模方式。实验显示，采用该注视分布的奖励模型在偏好预测准确率上高于基线模型，部分结果与人类阅读注意的既有发现定性一致，论文已被 NeurIPS 2026 主会接收。
正文 · 原文
Abstract:Integrating human feedback into language models has gained increasing attention as a means to align model outputs with human preferences, with Reinforcement Learning from Human Feedback (RLHF) serving as a prominent framework for preference-based alignment. Recent studies have incorporated eye-tracking (ET) data as an additional supervisory signal for reward model training, grounding reward learning in human reading behavior. Building on this approach, we extend prior fixation-centric models by leveraging established findings from cognitive psychology on human letter recognition. Specifically, we model visual attention as a spatially graded distribution centered on the current gaze location and incorporate this representation into reward model training. Our results show that the reward model trained with the proposed gaze distribution achieves higher preference prediction accuracy than a baseline model. Moreover, some results from our experiments are qualitatively consistent with established findings on human attention during reading, suggesting that they potentially capture cognitively meaningful aspects of human attention allocation.
| Comments: | 15 pages, 4 figures. Accepted to the 40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026 Main Track) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2611.13477 [cs.LG] |
| (or arXiv:2611.13477v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13477 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chaeho Lee [view email]
[v1]
Tue, 10 Nov 2026 02:36:14 UTC (2,079 KB)
来源：arXiv cs.CL · arxiv.org