arXiv cs.CL· Owen Walters, Lihu Chen, Francesca Toni·· 2 天前AI 评分44
检索到了却没读：搜索增强 LLM 为何用不上已有证据
Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence
AI 导读
研究发现搜索增强 LLM 常直接依据搜索摘要作答而不打开来源页面，这与更高的过时答案率和引用不实相关。团队用训练截止后发生变化的时间性事实，测试了一个商用模型的联网搜索和六个开放权重模型的受控检索配置，发现证据可用性与实际使用之间存在明显落差。即使检索到的证据已影响模型内部预测，最终答案仍可能偏向过时信息。
正文 · 原文
Abstract:Web search allows LLMs to access up-to-date information, but it remains unclear why they sometimes produce outdated answers even when relevant evidence is available. To study this problem, we construct temporal facts whose values changed after the models' training cutoffs. We evaluate one proprietary model under deployed web search and six open-weight model configurations under controlled retrieval. We find that models often answer from search snippets without opening the source pages, which is associated with higher rates of outdated answers and unfaithful citations. Our controlled experiments and layer-wise analysis reveal a gap between the availability of evidence and its effective use. Even when retrieved evidence influences internal predictions, models may still favour outdated information in their final answers. We hope these findings will help advance research on the reliable use of retrieved evidence in search-augmented LLMs.
| Comments: | 14 pages, 4 figures, 16 tables. Code and data: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.21731 [cs.CL] |
| (or arXiv:2610.21731v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21731 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Owen Walters [view email]
[v1]
Thu, 15 Oct 2026 11:48:37 UTC (142 KB)
来源：arXiv cs.CL · arxiv.org