arXiv cs.AI· Zijian Jia, Peixuan Zhang, Genying Li, Di Wu, Yujie Fu, Si Li, Boxin Shi, Shuchen Weng·· 12 小时前AI 评分31
LLaCO：面向大语言模型的竞争感知语义调度连续扩散框架
LLaCO: Continuous Diffusion for Large Language Models with Competition-Aware Semantic Scheduling
AI 导读
LLaCO 提出一种面向大语言模型的连续扩散生成框架，通过竞争感知语义调度解决连续轨迹中 token 身份衰减被压缩在噪声端点附近窄窗口的问题。该框架采用球面加噪与测地插值消除嵌入范数衰减，并按对数秩衰减比例重新分配去噪轨迹。以 8B 模型实例化后，LLaCO 超越此前连续扩散基线，并可与同规模离散扩散模型竞争。
正文 · 原文
Abstract:Continuous diffusion models have emerged as a promising non-autoregressive paradigm for text generation over continuous embedding spaces. Along the continuous trajectory, the residual token identity directly shapes the denoising objective. However, the geometric trajectory misaligns with temporal progression, triggering severe temporal compression. In this paper, we show that embedding norm decay causes Euclidean distance to yield spurious overtaking, indicating that token competition must be evaluated directionally. In this space, a candidate overtakes the source token only when the effective noise magnitude surpasses their semantic separation. We formalize these pairwise dynamics as the Competition Threshold Theory (CTT). Crucially, high-dimensional overtaking requires extreme noise levels, causing standard trajectories to concentrate identity decay into a narrow window near the noisy endpoint. To resolve this, we propose Semantic Retention Scheduling (SRS) to align time allocation with the progression of identity decay, and demonstrate that existing token identity metrics are fundamentally statistical aggregations of these pairwise crossings. Building on these theoretical insights, we introduce LLaCO, a generative framework powered by competition-aware semantic scheduling. LLaCO adapts a pretrained backbone to continuous representations via an embedding directional adapter and timestep-conditioned modulation. To operate within the directional space, LLaCO employs a spherical noising process formulated as a geodesic interpolation toward a mask-biased endpoint, eliminating norm decay while preserving accurate token competition. Guided by SRS, LLaCO reallocates the denoising trajectory proportionally to the logarithmic rank decay. Instantiated as an 8B model, LLaCO outperforms previous continuous diffusion baselines and rivals discrete diffusion models of comparable scale.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.32604 [cs.AI] |
| (or arXiv:2610.32604v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32604 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zijian Jia [view email]
[v1]
Fri, 23 Oct 2026 13:28:40 UTC (463 KB)
来源：arXiv cs.AI · arxiv.org