arXiv cs.CL· Amin Najafgholizadeh·· 12 小时前AI 评分38
Mix-FT：通过混合似然微调让掩码扩散语言模型实现少步采样
Train the Sampler You Run: Few-Step Masked Diffusion Language Models via Mixture-Likelihood Fine-Tuning
AI 导读
研究者提出混合似然微调（Mix-FT），在预训练模型上按其采样器似然的重要性加权界进行微调，无需教师模型、不改架构与采样器。在 LLaDA-8B 上，仅用预训练算力 0.002% 的 LoRA 微调，每块 16 步（模型评估量减半）的续写效果在三个评判器、各随机种子下均优于原模型，两位不同模型族的盲评 LLM 评审在 69% 和 71% 的样本对中更偏好其输出。
正文 · 原文
Abstract:When a masked diffusion language model is sampled in a few parallel steps in a random order, the model actually run is a mixture over reveal plans, while cross-entropy training fits token conditionals rather than this mixture. We introduce mixture-likelihood fine-tuning (Mix-FT), which fine-tunes a pretrained model on an importance-weighted bound of its own sampler's likelihood, with no teacher and no change to the architecture or the sampler. On LLaDA-8B, a LoRA fine-tune for 0.002% of the pretraining compute writes better OpenWebText continuations than continued cross-entropy and Alpha diffusion given the same compute, at every tested step count. With half the model evaluations, 16 rather than 32 steps per block, its continuations score better than the original model's in every seed under three judges, with confidence-based decoding as well as random order, and under confidence-based decoding two blinded LLM raters from different model families prefer them in 69 and 71% of pairs. On a public 110M-parameter block diffusion model, Mix-FT takes under four T4-hours and beats continued cross-entropy, Alpha diffusion and SDTT distillation at 4, 8 and 16 steps per block, under two judges in every seed. Exact likelihoods on text8, with a control that weights every plan equally, attribute about three quarters of the likelihood gain over continued cross-entropy to weighting each plan by how well it explains the sequence.
| Comments: | 41 pages, 8 figures, 34 tables |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.02666 [cs.LG] |
| (or arXiv:2611.02666v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02666 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Amin Najafgholizadeh [view email]
[v1]
Mon, 2 Nov 2026 07:24:22 UTC (262 KB)
来源：arXiv cs.CL · arxiv.org