arXiv cs.AI· Ke Li, Dong An, Xiaoling Zang, Can Ye, Liang Xie, Qibo Qiu, Chen Shen, Xiaofei He, Wenxiao Wang·· 13 小时前AI 评分32
InfoQuant：面向低比特 LLM 量化的激活分布设计方法
InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
AI 导读
InfoQuant 是一种免训练的低比特 LLM 量化方法，通过峰值抑制正交变换（PSOT）将激活塑造成更易离散化的分布，并引入自适应离群 token 选择提升优化鲁棒性。在 W4A4KV4 设置下，该方法平均保留 97% 的浮点精度，并将 LLaMA-2 13B 的性能差距较此前 SOTA 缩小 42%。
正文 · 原文
Abstract:Low-bit activation quantization remains a major bottleneck in efficient large language model (LLM) deployment. The difficulty is not only that activations contain outliers, but that their distributions are often poorly matched to a low-bit uniform quantizer. Existing post-training quantization (PTQ) methods suppress peaks, balance channels, or minimize reconstruction error, yet they rarely specify what activation distribution is actually easy to discretize. As a result, activations may appear numerically smoother while still incurring large quantization error because the quantization range remains wide or most values collapse into a few levels near the mean. We recast activation transformation as quantizer-facing distribution design and analyze quantization error from an information-theoretic perspective. Our analysis shows that quantization-friendly activations should jointly have a smaller numerical range and sufficient dispersion within that range. Guided by this analysis, we propose InfoQuant, a train-free method that employs Peak Suppression Orthogonal Transformation (PSOT) to shape activations into more quantization-friendly distributions. We further introduce adaptive outlier-token selection to improve the robustness of PSOT during optimization. Across multiple LLM families, InfoQuant consistently outperforms prior PTQ and end-to-end training baselines. Under W4A4KV4, it preserves 97% of floating-point accuracy on average and reduces the LLaMA-2 13B performance gap by 42% over the previous state of the art. Code is available at [this https URL](this https URL)
| Comments: | EMNLP2026 main |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.26175 [cs.LG] |
| (or arXiv:2605.26175v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.26175 arXiv-issued DOI via DataCite |
Submission history
From: Ke Li [view email]
[v1]
Mon, 25 May 2026 05:34:46 UTC (3,267 KB)
[v2]
Wed, 21 Oct 2026 14:39:26 UTC (3,267 KB)
来源：arXiv cs.AI · arxiv.org