arXiv cs.AI· Exauc\`e Luweh Adjim Ngarti (AIRSEA), \'Elise Arnaud (AIRSEA), L\'eo Nicoletti (Mines Saint-\'Etienne MSE, LIMOS, FAYOL-ENSMSE, FAYOL-ENSMSE), Victor Trappler (Mines Saint-\'Etienne MSE, LIMOS, FAYOL-ENSMSE, FAYOL-ENSMSE), Arthur Vidard (AIRSEA)·· 13 小时前AI 评分20
基于 Rényi 正则化流的变分推断用于贝叶斯校准
R\`enyi-Regularized Flow-Based Variational Inference for Bayesian Calibration
AI 导读
研究提出在基于归一化流的变分推断中加入 Rényi 散度正则项，以缓解贝叶斯模型校准中后验多峰导致的模式坍塌。在双参数孪生实验中，该正则化将近坍塌比例从 50% 降至 25%，连通双峰情形下流近似可覆盖两个模式。但该方法改变了贝叶斯变分目标，在近似精度上引入权衡，并非模式坍塌的通用解法。
正文 · 原文
Abstract:Variational inference with normalizing flows provides a flexible posterior approximation for Bayesian model calibration, but struggles when the posterior has several modes separated by regions of low probability mass. We study this behavior in a two-parameter twin experiment and regularize the variational objective. Under Gaussian observation errors and a Gaussian prior, the corresponding minimization objective is the expected Tikhonov calibration objective minus the differential entropy of the approximating distribution. However, the expected Tikhonov term favors concentration near one of the modes, while entropy maximization does not guarantee multimodality. We introduce a regularization term based on Rényi divergence that encourages the approximate posterior to cover the prior support. In the connected bimodal case, a flow approximation represents both modes; in the case with well-separated modes, the flow and Gaussian mixture frequently miss one mode. The introduced regularization reduces the near-collapse fraction from 50% to 25%. While the regularization improves mode coverage in this experiment, it changes the Bayesian variational objective, introduces a trade-off in approximation accuracy, and does not provide a general solution to mode collapse.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2610.37169 [cs.AI] |
| (or arXiv:2610.37169v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37169 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Exauce LUWEH ADJIM NGARTI [view email] [via CCSD proxy]
[v1]
Tue, 27 Oct 2026 12:57:10 UTC (859 KB)
来源：arXiv cs.AI · arxiv.org