arXiv cs.CL· Zhipin Wang, Koel Dutta Chowdhury, Christoph Leiter, Steffen Eger·· 11 小时前AI 评分31
检索文献如何影响 LLM 生成科研想法：五种检索策略对比研究
Tracing the Influence of Retrieved Literature on Scientific Ideation in LLMs
AI 导读
研究对比五种文献检索策略与无文献基线，在 15 个机器学习主题、十个开放权重模型上生成 45,000 个想法。相比无文献基线，五种检索策略使同一批论文生成想法的多样性下降 5.6%–17.9%，但混合不同论文集时多数策略使多样性提升 10.2%–18.1%。
正文 · 原文
Abstract:Large language models (LLMs) are increasingly used to generate research ideas from scientific literature. Existing systems combine literature retrieval with search, generation, and refinement, making it difficult to determine how the retrieval strategy itself shapes the ideas generated. We isolate this contribution by comparing five retrieval strategies and a No Literature baseline under a shared generation protocol, using 45,000 ideas across 15 machine learning topics and ten open-weight models. Relative to no literature basline, all five retrieval strategies reduce diversity among ideas generated from the same set of papers by $5.6\%$--$17.9\%$, while most increase diversity by $10.2\%$--$18.1\%$ when ideas from different sets are pooled. Effects on idea quality and novelty are modest on average but vary across topics. We further apply a JSD-based attribution method to trace the influence of individual input papers on each idea's motivation, core idea, approach, and validation. In $79.3\%$ of ideas, only one or two distinct papers receive the highest attribution scores across these four parts. These leading sources often recur across ideas generated from the same set of papers, and groups with more varied source attribution contain more diverse ideas ($r=0.70$). These findings motivate various sets of literature and the use of attribution to guide the choice of sources for subsequent ideas.
| Comments: | Updated preprint. Under review |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.39060 [cs.CL] |
| (or arXiv:2610.39060v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39060 arXiv-issued DOI via DataCite |
Submission history
From: Zhipin Wang [view email]
[v1]
Wed, 28 Oct 2026 23:45:23 UTC (1,041 KB)
[v2]
Fri, 30 Oct 2026 17:56:18 UTC (1,041 KB)
来源：arXiv cs.CL · arxiv.org