arXiv cs.AI· Rhea Sanjay Sukthanker, Riccardo Grazzi, James Hensman, Pashmina Cameron·· 12 小时前AI 评分34
超越 4 bits：以数据为中心的后训练量化与恢复方法
Beyond 4 bits: Data-Centric Post-training Quantization and Recovery
AI 导读
研究提出一种面向 2-bit 后训练量化（PTQ）的配方：采用向量码本表示、小型任务对齐校准数据，并将校准与恢复数据联合选择，在 3.8B 至 32B 的六个稠密与 MoE 模型上，以约 2.3 有效 bits/权重保留全精度 QA 准确率的 89% 至 97%，而标量 PTQ 已跌至随机水平。包含恢复阶段在内，其计算量比最强 QAT 基线少 1.4 至 30 倍。
正文 · 原文
Abstract:Post-training quantization (PTQ) compresses the weights of a trained model with limited compute and data. Down to 4 bits it is near-lossless, but at 2 bits scalar PTQ configurations fall to chance-level accuracy. We study three choices usually treated as implementation details that matter little at 4 bits but decide 2-bit quality: the quantization representation, the calibration data, and the recovery fine-tuning data, across six dense and mixture-of experts models (ranging from $3.8$ to $32$B parameters). Representation is the largest factor and vector codebooks restore usable models where scalar PTQ pipelines collapse. Most of the remaining loss in accuracy is on \emph{generative} tasks, and the amount of performance drop is model-dependent. Furthermore, at a short recovery budget, calibration and recovery data are coupled. The best recovery data depends on the calibration data, so the two must be chosen jointly. The resulting recipe involves a vector codebook representation, a small task-aligned calibration mixture, and an inexpensive codebook-only recovery stage. Including codebooks, it uses about $2.3$ effective bits per weight and retains $89$ to $97\%$ of full-precision question-answering (QA) accuracy where scalar PTQ is at chance, and, including recovery, uses $1.4$ to $30\times$ less compute than the strongest quantization-aware-training (QAT) baseline.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.28537 [cs.AI] |
| (or arXiv:2610.28537v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28537 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rhea Sanjay Sukthanker [view email]
[v1]
Tue, 20 Oct 2026 17:10:04 UTC (450 KB)
来源：arXiv cs.AI · arxiv.org