arXiv cs.CL· Koshiro Saito, Diasuke Oba, Danushka Bollegala, Naoaki Okazaki·· 12 小时前AI 评分31
扩散语言模型的公平性会随采样配置变化：LLaDA、LLaDA-1.5 与 Dream 在 Open-BBQ 上的评测
Beyond Model Weights: Sampling-Dependent Fairness in Diffusion Language Models
AI 导读
研究在 Open-BBQ 上评测 LLaDA、LLaDA-1.5 和 Dream 三个扩散语言模型，发现三者均存在社会偏见，且偏见幅度与模型间的偏见排序都高度依赖采样配置。针对自回归模型设计的去偏策略可有效迁移到扩散语言模型，在不损害下游性能的前提下降低偏见，但采样依赖性在去偏后依然存在。结果表明扩散语言模型的公平性无法脱离采样配置单独刻画。
正文 · 原文
Abstract:Diffusion language models (DLMs) are emerging as a practical alternative to autoregressive language models (ARLMs), yet their fairness remains largely uncharacterized. DLMs introduce sampling controls with no direct autoregressive counterpart, such as block length and remasking strategy, that directly set their speed--quality operating point. This raises a basic question: does a DLM's measured social bias remain stable across these settings? Across LLaDA, LLaDA-1.5, and Dream on Open-BBQ, we find that all three models exhibit social bias, but both the magnitude of measured bias and the relative ordering of models by bias are highly sensitive to the sampling configuration. This sampling dependence also changes what successful debiasing means. While mitigation strategies established for ARLMs transfer effectively to DLMs and reduce bias without degrading downstream performance, sampling dependence persists even after mitigation. Our results show that fairness in DLMs cannot be characterized independently of the sampling configuration, even after debiasing. Project page is available at this https URL.
| Comments: | 31 pages, 11 figures, 17 tables. Project page: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.27584 [cs.CL] |
| (or arXiv:2610.27584v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.27584 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Koshiro Saito [view email]
[v1]
Tue, 20 Oct 2026 03:49:20 UTC (594 KB)
来源：arXiv cs.CL · arxiv.org