arXiv cs.AI· Sungjun Lim, Adrian Robert Minut, Nico Carlsen Daheim, Mohammad Emtiyaz Khan, Kyungwoo Song, Thomas M\"ollenhoff·· 3 天前AI 评分24
用 Fenchel-Bregman 后验学习提升多样本推理：FBI 方法
Train for Many, Update with One: Improving Multi-Sample Reasoning via Fenchel-Bregman Posterior Learning
AI 导读
研究者提出有限 K 后验学习，训练求解器参数分布以降低 K 个独立候选全部失败的概率，并给出 Fenchel-Bregman（FB）方法，每次更新只需一次后验采样，而非直接蒙特卡洛所需的 K 次。FB 结合高斯后验变分优化器 IVON 得到 FBI，在多种递归与自回归推理任务上提升候选覆盖率，并以直接蒙特卡洛基线 12.3% 的实测单次更新训练时间达到相当的 Pass@10。
正文 · 原文
Abstract:Multi-sample reasoning measures candidate coverage, the probability that at least one of $K$ candidates is correct. However, minimizing the average loss of individual attempts does not directly optimize candidate coverage. We therefore formulate finite-$K$ posterior learning, which trains a distribution over solver parameters to reduce the probability that all $K$ independently generated candidates fail. To optimize this objective efficiently, we propose a Fenchel-Bregman (FB) approach that requires only one posterior draw per update, rather than the $K$ used by direct Monte Carlo optimization. FB first uses an exact Fenchel reformulation to separate problem-specific weights from the posterior update, enabling one-draw posterior updates. It then applies a closed-form Bregman update to track these weights through an exponential running average of observed failures, reusing the same draw. We instantiate FB with IVON, a variational optimizer for Gaussian posteriors over solver parameters, yielding Fenchel-Bregman IVON (FBI). Across diverse recursive and autoregressive reasoning tasks, FBI improves candidate coverage. In a matched comparison, it achieves comparable Pass@10 using 12.3% of the direct Monte Carlo baseline's measured training time per update.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.17735 [cs.LG] |
| (or arXiv:2610.17735v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17735 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sungjun Lim [view email]
[v1]
Tue, 13 Oct 2026 05:13:26 UTC (522 KB)
来源：arXiv cs.AI · arxiv.org