研究将 LLM 的科学发现能力追溯到其训练数据

Tracing the Scientific Discovery Capacity of LLMs to Their Training Data

AI 导读

一项研究提出用"缺失概率质量"框架解释 LLM 的科学发现能力，认为训练中仅出现一次的"单次事实"（monofact）决定发现能力。在材料科学与调控基因组学中直接操纵训练集单次事实比例后，两个开源权重模型族、多种采样策略与推理预算下均表现为单次事实越多发现越多。该关系无法仅用训练集多样性解释，且配合领域内数据时可跨科学领域迁移。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large language models (LLMs) show growing promise for scientific discovery, yet basic questions remain about what enables reliable discovery. How can we trace the scientific discovery capacity of LLMs to their training data? We study this question as one of missing probability mass and develop a theoretical account in which monofacts-the facts observed once during training-set the discovery capacity. In materials science and regulatory genomics, we directly manipulate monofact rates in training sets and find that more monofacts consistently yield more discoveries across two open-weight model families, sampling strategies, and reasoning budgets. Further experiments show the relationship cannot be explained by training-set diversity alone and can transfer across scientific domains when paired with in-domain data. Together, our results identify monofact rate in training data as a key factor governing discoveries in language models.

Subjects:	Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2611.03617 [cs.CL]
	(or arXiv:2611.03617v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2611.03617 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Arda Uzunoglu [view email]
[v1] Mon, 2 Nov 2026 18:50:54 UTC (271 KB)

来源：arXiv cs.CL · arxiv.org