arXiv cs.AI· DiffuSpace Technical Team·· 10 小时前AI 评分60
Clover-35B-A3B-Preview 技术报告:35B 总参数 MoE 扩散语言模型
Clover-35B-A3B-Preview Technical Report
AI 导读
DiffuSpace 团队发布 Clover-35B-A3B-Preview 技术报告,这是一个总参数 35B、激活参数 3B 的 MoE 扩散语言模型,支持 thinking 模式生成。
正文 · 原文
Abstract:We introduce Clover-35B-A3B-Preview (Clover), a mixture-of-experts diffusion language model with 35 billion total parameters and 3 billion activated parameters, supporting thinking-mode generation. Clover adapts Gated Delta Networks to block diffusion by combining recurrent memory of previously generated context with bidirectional softmax attention inside the current block. Historical memory readout competes with current-block tokens in a shared softmax, with learnable block-size calibration and a separate, zero-initialized right-context convolution branch. Clover leads the seven evaluated agent metrics, including 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1. On internal agent workloads, it reaches a reported generation speed of 680 tokens/s using one NVIDIA GB300 GPU at BF16 precision and batch size one. We organize the evaluation around model quality, drafting and verification costs, and deployment under constrained memory. We also describe a decision-model variant that generates answers to related questions in one denoising pass. On an internal set of 500 groups, joint prediction raises question accuracy from 90.37% to 93.88% and all-correct group accuracy from 65.4% to 77.0%.
| Comments: | Technical Report. Code: this https URL Weights: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.24636 [cs.CL] |
| (or arXiv:2611.24636v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24636 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shansan Gong [view email]
[v1]
Wed, 18 Nov 2026 15:46:14 UTC (423 KB)
来源:arXiv cs.AI · arxiv.org