arXiv cs.AI· Anita Kriz, Antonin Berthon, Nicol\'as Astorga, Mihaela van der Schaar·· 2 天前AI 评分24
将序列贝叶斯实验设计摊销进推理 LLM
Amortizing Sequential Experimental Design in Reasoning LLMs
AI 导读
研究将序列贝叶斯实验设计（BED）摊销进 LLM，通过信息增益奖励显式训练引导动作选择的推理过程，实验生成模型独立于 LLM 自身信念。在 Twenty Questions 上，所得策略提升信息获取与任务表现，以显著更低计算成本超越推理时和训练时 BED 基线，并泛化到未见类别。
正文 · 原文
Abstract:Despite their extensive pretraining knowledge and strong reasoning capabilities, large language models (LLMs) struggle to adaptively gather information in experimental design tasks. We study how to amortize sequential Bayesian experimental design (BED) into an LLM by explicitly training the reasoning that guides action selection using information-gain rewards. Our approach specifies the experimental generative model independently of the LLM's own beliefs, allowing us to optimize expected information gain under a model of our choosing. We investigate the design choices for amortization, including alternative generative-model constructions, reward estimation, hypothesis-space specification, multi-step objectives, and computational costs relative to training- and inference-time BED baselines. On Twenty Questions, the resulting policies improve information acquisition and task performance, outperform inference- and training-time BED baselines at substantially lower computational cost, and generalize to unseen categories. We further demonstrate positive transfer from Twenty Questions training to Medi-Q, a real-world clinical dialogue benchmark. These results suggest that information gathering can be learned as a generalizable reasoning skill in LLMs.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.20832 [cs.LG] |
| (or arXiv:2610.20832v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20832 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Antonin Berthon [view email]
[v1]
Wed, 14 Oct 2026 19:15:58 UTC (1,676 KB)
来源：arXiv cs.AI · arxiv.org