arXiv cs.AI· Mahesh Bhosale, Yuanhao Zhai, Siwei Lyu, Junsong Yuan, David Doermann, Defang Chen·· 12 小时前AI 评分31
FisherDistill：用闭式 Fisher 判别器实现扩散分布蒸馏
FisherDistill: Diffusion Distribution Distillation via Closed-Form Fisher Critics
AI 导读
FisherDistill 提出用 Fisher 线性判别分析替代可学习对抗判别器，完成扩散模型蒸馏中的分布精修。方法利用冻结教师模型的中间表示估计真实与生成样本的一阶、二阶统计量，导出闭式 Fisher 方向以最大化分离特征分布，无需额外判别器及其优化循环。在多个蒸馏框架上，该方法均优于各自蒸馏基线，并大幅缩小与对抗精修版本的差距。
正文 · 原文
Abstract:Recent advances in diffusion distillation have enabled high-quality image generation in only one or a few network function evaluations (NFEs). Despite their efficiency, aggressively distilled generators often retain a residual distribution gap from their multi-step teachers, motivating recent methods to introduce adversarial refinement with a learned discriminator. In this work, we ask whether such refinement necessarily requires a learned adversarial discriminator. We introduce \textbf{FisherDistill}, a simple alternative based on Fisher linear discriminant analysis. Using intermediate representations from the frozen teacher, we estimate the first- and second-order statistics of real and generated samples and derive a closed-form Fisher direction that maximally separates their feature distributions relative to within-class variation. This direction provides a lightweight refinement signal that complements the original distillation objective without requiring an additional adversarial discriminator or its optimization loop. Across multiple distillation frameworks, our approach consistently improves generation quality over their respective distilled baselines and substantially closes the gap to adversarially refined counterparts. Code: this https URL
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.07604 [cs.CV] |
| (or arXiv:2611.07604v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07604 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mahesh Bhosale Mr [view email]
[v1]
Thu, 5 Nov 2026 07:53:14 UTC (5,846 KB)
来源：arXiv cs.AI · arxiv.org