arXiv cs.CL· Zhe Xu, Mingfu Liang, Kavosh Asadi, Jean-Pascal Billaud, Tianyi Chen, Yanhui Guo, Christina Huang, Baekjin Kim, Jimmy Kim, Yufei Li, Daqi Sun, Jiachen Sun, Dian Yu, Zaiwei Zhang, Kaushik Rangadurai, Frank Shyu, Sandeep Pandey, Xi Liu, Luke Simon·· 2 天前AI 评分20
排列扩散 LLM 的设计空间探索
Exploring the Design Space of Permutation Diffusion LLMs
AI 导读
研究通过三种扩散 LLM 参数化方法解决排列生成问题：基于打乱候选索引序列的硬分配预测、基于双随机矩阵的软分配预测，以及直接预测排序分数后排序。在三个 Amazon 领域的推荐重排序任务（候选集大小 10 和 20）上，单次函数评估时硬分配和软分配变体的 Recall@1 与受限掩码扩散基线相当，而直接排序分数预测无需匈牙利算法或 Sinkhorn 归一化即可保证有效排列。
正文 · 原文
Authors:Zhe Xu, Mingfu Liang, Kavosh Asadi, Jean-Pascal Billaud, Tianyi Chen, Yanhui Guo, Christina Huang, Baekjin Kim, Jimmy Kim, Yufei Li, Daqi Sun, Jiachen Sun, Dian Yu, Zaiwei Zhang, Kaushik Rangadurai, Frank Shyu, Sandeep Pandey, Xi Liu, Luke Simon
Abstract:Permutation generation with language models requires both text-conditioned ranking and a global one-to-one assignment between candidates and output positions. We study this problem through three diffusion-LLM parameterizations: hard-assignment prediction over shuffled candidate-index sequences, soft-assignment prediction over doubly stochastic matrices, and direct ordering-score prediction followed by sorting. The three families place permutation constraints at different stages of prediction and decoding and therefore support different training objectives and sampling procedures. We evaluate these designs on recommendation reranking across three Amazon domains with candidate sets of size 10 and 20. At one function evaluation, the hard- and soft-assignment variants achieve Recall@1 comparable to a constrained masked-diffusion baseline, whereas direct ordering-score prediction guarantees valid permutations without Hungarian assignment or Sinkhorn normalization. Additional function evaluations do not consistently improve ranking quality, and their effect depends strongly on the state representation and training objective. These results show that corruption, prediction parameterization, objective, and decoder should be designed jointly for permutation diffusion with language-model denoisers.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.20635 [cs.CL] |
| (or arXiv:2610.20635v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20635 arXiv-issued DOI via DataCite |
Submission history
From: Zhe Xu [view email]
[v1]
Thu, 1 Oct 2026 03:39:40 UTC (472 KB)
来源：arXiv cs.CL · arxiv.org