arXiv cs.AI· Peng Tu, Baoyou Chen, Haoyuan Xia, Zilong Dong, Weihao Yuan, Siyu Zhu·· 11 小时前AI 评分31
BCSM：面向离散扩散模型蒸馏的双向中心化分数匹配
Bidirectional Centered Score Matching for Discrete Diffusion Distillation
AI 导读
针对离散扩散模型少步蒸馏中 token-wise KL 监督不足的问题，研究者提出双向中心化分数匹配（BCSM），用 O(K) 双向活跃集把成对比较降至 O(K²) 并分解为 O(K) 目标，同时引入跨时间步转移对齐。实验显示单步推理下 NFE 最多减少 6 倍，HumanEval 相对提升 16.9%，MMMU 相对提升 15.1%。
正文 · 原文
Abstract:Discrete diffusion models with parallel decoding are often distilled using token-wise KL objectives. Although token-wise KL exactly matches the corresponding product-form block distributions, its probability-space optimization geometry and teacher-only candidate truncation may provide insufficient supervision under aggressive few-step distillation. Concrete Score Matching (CSM) provides an alternative within-position objective by aligning relative logit geometry, but its complete pairwise form incurs \mathcal{O}(|\mathcal{V}|^2) complexity. We propose Bidirectional Centered Score Matching (BCSM), an efficient within-position distillation objective that constructs an \mathcal{O}(K) bidirectional active set, reduces pairwise comparisons to \mathcal{O}(K^2), and factorizes them into an \mathcal{O}(K) objective. We further introduce cross-timestep transition alignment. Experiments demonstrate strong quality--efficiency trade-offs, including up to 6\times NFE reduction with a 16.9\% relative HumanEval gain and a 15.1\% relative MMMU gain at single-step inference.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.04279 [cs.AI] |
| (or arXiv:2611.04279v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.04279 arXiv-issued DOI via DataCite |
Submission history
From: Yh.Peng Tu [view email]
[v1]
Tue, 3 Nov 2026 04:05:41 UTC (7,225 KB)
[v2]
Wed, 4 Nov 2026 03:21:50 UTC (7,225 KB)
来源：arXiv cs.AI · arxiv.org