arXiv cs.AI· Sunghyun Wee, Geunjae Choi, Hyeonjin Kim, Suyoung Kim, Nojun Kwak·· 2 天前AI 评分27
FPoT：面向高效无乘法器 LLM 推理的分数次幂量化
Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
AI 导读
研究者提出 FPoT，一种基于 √2（半倍频程）的分数次幂网格量化方法，可在保持全整数移位加法数据通路的同时实现实用的 4-bit 权重量化。在 Llama-3-8B 的 4-bit 权重与激活量化下，FPoT 相比 PoT 基线将零样本常识推理性能提升 8.72%，与均匀整数基线仅差 0.4 个百分点。
正文 · 原文
Abstract:Large Language Models (LLMs) demand efficient inference, where the multiply-accumulate (MAC) operations in linear projections dominate compute and energy cost. Post-training quantization (PTQ) reduces this cost by mapping weights and activations to low-bit integers. Power-of-Two (PoT) weight quantization promises efficient multiplier-free LLM inference via shift-add logic, but suffers significant accuracy degradation at low bitwidths due to its coarse base-2 quantization grid. We introduce Fractional PoT (FPoT), a base-$\sqrt{2}$ (half-octave) fractional PoT grid that achieves practical 4-bit weight quantization while preserving a fully integer shift-add datapath. The method combines (i) the base-$\sqrt{2}$ weight grid with uniform integer activations, (ii) a calibration-time grid-alignment refinement absorbed into the static per-channel scale, and (iii) a lightweight 1-adder $\sqrt{2}$ approximation whose error is absorbed during weight calibration via approximate-grid quantization, yielding a processing element (PE) with significantly fewer gates than a 4-bit integer multiplier. On Llama-3-8B with 4-bit weight and activation quantization, FPoT improves zero-shot commonsense reasoning performance by 8.72% relative to the PoT baseline, while maintaining performance close to the uniform integer baseline, with only a negligible 0.4%p degradation. These results hold across various configurations spanning seven models and three bit-widths. Furthermore, FPoT can be integrated with state-of-the-art LLM PTQ methods, demonstrating framework-agnostic applicability.
| Comments: | 37 pages, 19 figures, 22 tables. Accepted at NeurIPS 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.21257 [cs.LG] |
| (or arXiv:2610.21257v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21257 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sunghyun Wee [view email]
[v1]
Thu, 15 Oct 2026 04:50:03 UTC (301 KB)
来源：arXiv cs.AI · arxiv.org