arXiv cs.AI· Harshit Samrat, Ramakrishna Paramahamsa Kompella·· 4 小时前AI 评分34
MoE 扩散解码的隐藏成本:剖析并修复采样器
The Hidden Cost of MoE Diffusion Decoding: Profiling and Fixing the Sampler
AI 导读
对 26B 参数扩散 MoE 模型在 NVIDIA GB10 上的剖析显示,权重量化后 logits 采样与停止流水线而非专家 GEMM 成为解码最大耗时来源,因为扩散解码每个输出 token 需处理 S 倍(S 为去噪步数)的 logit 条目。
正文 · 原文
Abstract:Masked diffusion language models (dLLMs) decode by iteratively denoising a full canvas of tokens, so every forward pass runs a per-token sampling and stopping pipeline over the entire canvas-by-vocabulary logits tensor. We profile single-user decoding of a 26B-parameter diffusion Mixture-of-Experts model on a bandwidth-bound edge accelerator (NVIDIA GB10) and report a result that is easy to miss: after weight quantisation, this logits pipeline, not the expert GEMMs, is the single largest consumer of decode time. Unlike autoregressive decoding, where sampling touches one position's logits per committed token, diffusion decoding processes S times as many logit entries per output token, where S is the number of denoising steps. We show this pipeline is largely redundant or fusable and remove it in two lossless tiers: a bit-exact tier whose output is byte-identical to the reference decoder, and an accuracy-parity tier using a fused single-pass entropy kernel and Gumbel-max sampling. Combined with a W4A16 expert kernel, the full stack reaches 2.4-2.7 times end-to-end speedup over bf16 at unchanged benchmark accuracy on GSM8K, HumanEval, and MATH, on a single 273 GB/s device. We confirm the finding on a second, independently built diffusion MoE (LLaDA2.0-mini: pipeline 28.8% of decode, fused W4A16 3.3 times at accuracy parity) and show it disappears on a dense diffusion model (3.6%). The cost is specific to MoE diffusion, where quantisation makes the backbone cheap. Code and quantised weights are released.
| Comments: | Accepted to NeurIPS workshop DiffuLM 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.21852 [cs.LG] |
| (or arXiv:2611.21852v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21852 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ramakrishna Paramahamsa Kompella [view email]
[v1]
Mon, 16 Nov 2026 20:16:09 UTC (15 KB)
来源:arXiv cs.AI · arxiv.org