arXiv cs.AI· Jialu Pan, Yufeng Zhang, Nan Hu, Zhenbang Chen, Ji Wang, Keqin Li·· 12 小时前AI 评分18
KL 散度在高斯分布扰动下的最坏情况最优 √ε 速率稳定性
Worst-Case Optimal $\sqrt{\epsilon}$-Rate Stability of KL Divergence under Perturbations of Gaussian Distributions
AI 导读
研究证明：当高斯参考分布被扰动时，KL 散度满足 √ε 速率的最坏情况最优稳定性界。设 N1、N2 为 d 维高斯分布，P 为任意满足 E_P[‖X‖²]<∞ 且 KL(P‖N1)<∞ 的分布，若 KL(N1‖N2)≤ε（0<ε<1/12），则 |KL(P‖N1)−KL(P‖N2)| 的上界为 √(6dε)/2 + K√ε + o(√ε)。
正文 · 原文
Abstract:We establish a stability bound for the Kullback-Leibler (KL) divergence when its Gaussian reference distribution is perturbed. Existing relaxed triangle inequalities of KL divergence between Gaussian distributions require all three distributions to be Gaussian, which limits their use in applications such as out-of-distribution detection with flow-based generative models. Let $\mathcal{N}_1$ and $\mathcal{N}_2$ be $d$-dimensional Gaussian distributions, and let $P$ be any $d$-dimensional distribution satisfying $\mathbb{E}_P[\|X\|^2]<\infty$ and $\mathrm{KL}(P\Vert \mathcal{N}_1)<\infty$. If $\mathrm{KL}(\mathcal{N}_1\Vert\mathcal{N}_2)\leq\epsilon$ for $0<\epsilon<1/12$, we give an upper bound of $\left|\mathrm{KL}(P\Vert\mathcal{N}_1)-\mathrm{KL}(P\Vert\mathcal{N}_2)\right|$. Under suitable uniform boundedness conditions, the bound is $ \frac{\sqrt{6d\epsilon}}{2}+K\sqrt{\epsilon}+o(\sqrt{\epsilon})$ as $\epsilon\rightarrow 0$ for fixed $d$, where $K$ depends on the parameters of $\mathcal{N}_1$ and $\mathcal{N}_2$, and on $\mathbb{E}_P[\|X\|]$ and $\mathbb{E}_P[\|X\|^2]$. Furthermore, we prove that the $\sqrt{\epsilon}$ dependence is worst-case optimal. The result extends stability guarantees restricted to Gaussian triples to comparisons involving an arbitrary distribution and two nearby Gaussian references. Experiments on synthetic distributions and latent representations from flow-based models support the predicted stability behavior.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.11026 [cs.LG] |
| (or arXiv:2604.11026v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.11026 arXiv-issued DOI via DataCite |
Submission history
From: Yufeng Zhang [view email]
[v1]
Mon, 13 Apr 2026 05:49:59 UTC (1,186 KB)
[v2]
Wed, 15 Apr 2026 09:15:32 UTC (1,188 KB)
[v3]
Thu, 16 Apr 2026 15:03:23 UTC (1,275 KB)
[v4]
Fri, 16 Oct 2026 15:32:20 UTC (13,681 KB)
来源：arXiv cs.AI · arxiv.org