arXiv cs.AI· Mahsa Salmani, Sayeh Sharify, Hesham Mostafa·· 6 天前AI 评分31
DualQuant：面向 LLM 的双尺度免校准权重与激活量化
DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs
AI 导读
DUALQUANT 是一个双尺度、免校准的 LLM 训练后量化框架，通过最小化权重重建 MSE 直接得到逐通道激活缩放因子，无需校准数据即可完成激活量化。该方法是首个正式支持 MX 全家族与 NVFP4 等块格式的双尺度量化算法，在 Llama 和 Qwen 上于 WikiText-2、C4 及零样本任务中匹配或超越基于校准的 SmoothQuant。
正文 · 原文
Abstract:Low-bit inference on accelerators with native 4-bit support requires quantizing both weights and activations to realize the full throughput gains these formats provide. While post-training weight quantization has been extensively studied, activation quantization remains considerably more challenging and typically requires calibration data. In this work, we introduce DUALQUANT, a dual-scale, calibration-free post-training quantization framework that directly minimizes the weight-reconstruction Mean Squared Error (MSE) and, as an immediate by-product, yields per-channel activation rescalings prior to activation quantization. Our key observation is that the scaling factors obtained by minimizing weight-reconstruction error provide effective activation rescalings, enabling competitive activation quantization without calibration data. We formulate DUALQUANT as a non-differentiable weight-reconstruction problem and develop an efficient solver based on variable splitting and block coordinate descent, where each subproblem admits a closed-form update. DUALQUANT provides a unified algorithm supporting a wide range of formats, including the full MX family and NVFP4. To our knowledge, DUALQUANT is the first dual-scaling method to formally support MX and NV block formats. Across Llama and Qwen models evaluated on WikiText-2, C4, and zero-shot downstream tasks, DUALQUANT matches or exceeds the accuracy of calibration-based SmoothQuant across most models and formats, and substantially outperforms calibration-free baselines, with large gains for MX and NV block formats. We validate these gains with a real packed W4A4 inference path for Llama-3.1-8B on an RTX 4090. The resulting implementation closely matches the accuracy of fake quantization while delivering up to 2.61x memory reduction and up to 2.46x throughput speedup over BF16.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.13767 [cs.LG] |
| (or arXiv:2610.13767v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13767 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mahsa Salmani [view email]
[v1]
Fri, 9 Oct 2026 15:51:38 UTC (940 KB)
来源：arXiv cs.AI · arxiv.org