Johns Hopkins 论文发现训练数据中单次出现的事实越多 LLM 科学发现率越高
Johns Hopkins 新论文发现，LLM 训练数据中只出现过一次的事实（monofacts）占比越高，科学发现率越高。作者在 gpt-oss-20b 和 gemma-3-12b-it 上分别用材料与 DNA 数据集微调，将 monofact 占比从 0% 调到 100%，以精确还原训练中留出的材料或 DNA 序列作为发现评分，发现率在每个设置下都随占比上升。
Rohan Paul· @rohanpaul_ai · X·2026-11-06 06:24· 58 分钟前AI 评分58
AI 导读
Johns Hopkins 新论文发现，LLM 训练数据中只出现过一次的事实（monofacts）占比越高，科学发现率越高。
正文 · 原文
Strange findings here.
New Johns Hopkins paper finds that LLMs make more scientific discoveries when more of their training facts appear only once.
discovery rates rose steadily as the share of once-seen facts in training went from 0% to 100%, for 2 model families.
Discovery and hallucination both mean producing something the model never saw in training. Earlier theory tied hallucination to "monofacts", facts seen exactly once, and treated that unseen space as mostly false.
The authors extend that theory to show monofacts also set how much a model can discover. They fine-tuned gpt-oss-20b and gemma-3-12b-it on materials and DNA datasets, varying the monofact share from 0% to 100%.
Discovery, scored as exactly recovering materials or DNA sequences held out from training, rose steadily with the monofact share in every setting. Training for 3 or 5 epochs, which turns single sightings into repeats, made the trend disappear.
When building science models, keep rare one-off examples and avoid repeating data, and expect more hallucinations to come with more discoveries.
来源：Rohan Paul · x.com
来源：RadarAI · aihot.news