arXiv cs.CL· Xingyu Liu, Xiangyang Yin, Tianhua Xia, Haiyu Wang, Sai Qian Zhang·· 11 小时前AI 评分26
CodeQuant2：面向低精度大推理模型的离群值缓解与自适应聚类量化框架
CodeQuant2: Enhanced Outlier Mitigation and Adaptive Clustering for Low-Precision LRMs
AI 导读
CodeQuant2 是一个基于聚类的 LUT 量化框架，用于低精度大推理模型（LRM），通过旋转与缩放缓解激活离群值、用聚类表示非均匀权重分布，并按各层量化难度自适应调整码本大小。该框架还改进了校准样本选择，以匹配推理解码早期的激活波动，配合定制 LUT CUDA kernel 在推理基准上显著提升精度，相对 BF16 最高实现 2.85 倍加速。
正文 · 原文
Abstract:Large reasoning models (LRMs) improve task performance by generating longer intermediate reasoning traces, but this substantially increases inference costs and makes low-precision quantization increasingly important. However, aggressively quantized models often perform poorly on reasoning tasks. We observe larger activation fluctuations during the first decoding steps of reasoning. Calibration that omits these early states may miss their activation fluctuations. To address these reasoning-specific quantization challenges, we propose \textbf{CodeQuant2}, a clustering-based LUT quantization framework for low-precision large reasoning models. CodeQuant2 mitigates activation outliers with rotation and scaling, uses clustering to represent non-uniform weight distributions, and adapts codebook sizes to the quantization difficulty of each layer. Guided by our observation of reasoning-time activation behavior, CodeQuant2 further improves calibration sample selection to better match actual decoding statistics. Combined with our customized LUT CUDA kernel, CodeQuant2 delivers substantial accuracy improvements on reasoning benchmarks while achieving up to $2.85\times$ speedup over BF16.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.09868 [cs.CL] |
| (or arXiv:2611.09868v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.09868 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiangyang Yin [view email]
[v1]
Fri, 6 Nov 2026 18:48:04 UTC (659 KB)
来源：arXiv cs.CL · arxiv.org