跳到正文
arXiv cs.CL· Artem Vazhentsev, Vladislav Smirnov, David Li, Maxim Panov, Timothy Baldwin, Artem Shelmanov·· 9 小时前AI 评分34

面向大型语言扩散模型的不确定性量化研究

Uncertainty Quantification for Large Language Diffusion Models

AI 导读

研究首次系统探讨大型语言扩散模型(LLDMs)的不确定性量化,提出从迭代去噪过程提取轻量级零样本不确定性信号,并结合掩码扩散似然与轨迹语义差异改进现有方法。在三个任务、八个数据集和四个模型上的实验显示,该方法接近最强采样基线,计算开销最多降低 100 倍。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large Language Diffusion Models (LLDMs) are emerging as an alternative to autoregressive models, offering faster inference through higher parallelism. Similar to autoregressive LLMs, they remain prone to hallucinations, making reliable uncertainty quantification (UQ) crucial for safe deployment. However, existing UQ methods are fundamentally misaligned with this new paradigm: they assume autoregressive factorization or use expensive repeated sampling, negating the efficiency of LLDMs. In this work, we present the first systematic study of UQ for LLDMs and propose lightweight, zero-shot uncertainty signals derived from the iterative denoising process, leveraging intermediate generations, token remasking dynamics, and denoising complexity. We further adapt a state-of-the-art UQ method to LLDMs by combining masked diffusion likelihoods with trajectory-based semantic dissimilarity. We prove that expected trajectory dissimilarity lower bounds the masked diffusion training objective, which motivates its usage as an uncertainty score. Comprehensive experiments across three tasks, eight datasets, and four models show that our method achieves a great cost-performance trade-off: it approaches the strongest sampling-based baselines while incurring up to 100x lower computational overhead. Our work demonstrates that LLDMs can deliver both fast inference and reliable hallucination detection simultaneously.
Comments: Advances in Neural Information Processing Systems 39 (NeurIPS 2026)
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2605.14570 [cs.CL]
  (or arXiv:2605.14570v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2605.14570

arXiv-issued DOI via DataCite

Submission history

From: Artem Vazhentsev [view email]
[v1] Thu, 14 May 2026 08:39:56 UTC (853 KB)
[v2] Tue, 17 Nov 2026 13:27:44 UTC (2,041 KB)

来源:arXiv cs.CL · arxiv.org