arXiv cs.AI· Wang Ru·· 13 小时前AI 评分34
高斯先验何时失效？RL 微调的潜在几何研究
When Do Gaussian Priors Fail? A Latent Geometry Investigation of RL Fine-Tuning
AI 导读
研究发现，潜扩散模型普遍采用的高斯先验在 RL 微调下存在结构性缺陷：标准 VAE 的 KL 项会惩罚模态分离，多模态数据被压缩进重叠的潜空间。在真实丙氨酸二肽玻尔兹曼分布上，高斯先验 VAE 在 beta_c = 0.224 时崩溃，而模态追踪的高斯混合先验在 0.00316 时崩溃，同一流程下相差 71 倍。
正文 · 原文
Abstract:Latent diffusion models almost universally place a Gaussian prior on the VAE latent space. For base generation this default is unproblematic; under reinforcement-learning fine-tuning, however, it carries a structural limitation. The standard VAE's KL term does not merely ignore mode structure - it actively taxes separation and tightness. When data are multimodal, the ELBO optimum compresses distinct modes into an overlapping latent. RL fine-tuning amplifies this overlap: the score network is under-resolved on the saddle region between modes, dynamic-to-variational amplification rises sharply, and the collapse boundary beta_c is pushed far above its variational scale. On the real alanine-dipeptide Boltzmann distribution, a Gaussian-prior VAE collapses at beta_c = 0.224 while a mode-tracked Gaussian-mixture prior collapses at 0.00316, a 71x gap from the same pipeline.
We trace this gap to a single geometric mediator, d_min/(2 sigma_0), and verify the causal chain through controlled interventions: moving geometry at fixed prior, and switching prior at matched geometry (full mediation; prior-label coefficient 0.995x, 95% CI [0.79, 1.25]). A three-source decomposition localises the amplification to the learned policy (zero ODE-solver contribution, bounded anchor mismatch); the coverage plateau is an unconditional benefit of mode-separating priors that no amount of RL capacity recovers. The remedy is conditional: a non-Gaussian prior is necessary precisely when the latent geometry would fall into overlap and the task is coverage-sensitive. A machine-checked theory skeleton supports the KL pricing structure - a divergent compactness tax under a Gaussian prior versus a finite assignment fee under a mixture - and the position-versus-plateau distinction delimits what is finite-resource from what requires changing the prior.
| Comments: | around 49 pages, around 25 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| MSC classes: | 68T07, 62F15 |
| ACM classes: | I.2.6; G.3 |
| Cite as: | arXiv:2610.28849 [cs.LG] |
| (or arXiv:2610.28849v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28849 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rui Wang [view email]
[v1]
Tue, 20 Oct 2026 19:22:06 UTC (2,763 KB)
来源：arXiv cs.AI · arxiv.org