arXiv cs.CL· Yu Hou, Hal Daum\'e III, Rachel Rudinger, William Walden·· 13 小时前AI 评分31
论文研究引用在偏好数据中的作用：人类偏好更多样但更少的引用，LLM 存在引用相关偏好
On the Role of Citations in Preference Data
AI 导读
一项 EMNLP 2026 论文研究引用在科学问答偏好判断中的作用，发现人类偏好更多样但总体更少的引用，而四个开源 LLM 在无法访问来源的情况下仍表现出部分引用相关偏好，且这些偏好取决于数据和具体模型。研究使用混合效应模型分析引用对成对判断的影响，并讨论了对偏好数据收集的启示。
正文 · 原文
Abstract:Many NLP tasks require systems to provide attribution in their outputs--i.e. citations to grounding sources. Attribution serves as a bulwark against model hallucination and as a means for users to verify the credibility of model outputs. Yet, it is unclear how humans and LLMs evaluate citations when comparing outputs, a process central to reward modeling and modern LLM post-training. This paper studies the role of citations in the preferences of human judges and four open-source LLMs within the context of scientific question answering, leveraging mixed effects models to investigate the influence of citations on pairwise judgments. Among our key findings are (1) that humans prefer more diverse citations but fewer overall, and (2) that LLMs show some citation-related preferences compared to humans, despite lacking access to the sources, but these preferences depend on the data and specific models. We further discuss the implications of our findings for preference data collection.
| Comments: | EMNLP 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.21376 [cs.CL] |
| (or arXiv:2608.21376v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.21376 arXiv-issued DOI via DataCite |
Submission history
From: Yu Hou [view email]
[v1]
Mon, 6 Jul 2026 18:17:27 UTC (1,441 KB)
[v2]
Wed, 21 Oct 2026 05:52:57 UTC (1,442 KB)
来源：arXiv cs.CL · arxiv.org