arXiv cs.AI· Ning Yang, Bohan Cui, Zhuowen Liu, Yuting Wang·· 13 小时前AI 评分31
ResCom：用激活残差补偿提升大语言模型量化精度
ResCom: Activation Residual Compensation for Large Language Model Quantization
AI 导读
ResCom 提出一种低秩校正方法，利用激活舍入残差补偿量化后的层输出误差，通过输出感知校准以闭式解选取残差方向，无需梯度训练或更新已校准权重。
正文 · 原文
Abstract:Low-bit weight--activation quantization reduces the memory and computation needed to deploy large language models, but activation rounding can still degrade accuracy after calibration. Reconstruction from rounded inputs cannot distinguish activations within the same quantization bin. We propose ResCom, a low-rank correction that uses the activation rounding residual to compensate for the resulting layer-output error. An output-aware calibration procedure selects residual directions by their contribution to output reconstruction and computes the correction in closed form, without gradient training or updates to the calibrated weights. On Llama-3-8B with 4-bit weights and activations, rank-32 attention corrections reduce WikiText-2 perplexity from 7.708 to 7.147, improve MMLU by 2.89 percentage points, and raise five-shot GSM8K accuracy from 23.96 to 36.16 percent. Evaluations on models from 1B to 70B parameters and on held-out C4 test the method beyond this setting. The evaluated FP32 implementation adds 2.25 percent to 4K prefill time; a separate compact-storage implementation retains a 27.7 percent peak-memory reduction relative to FP16.
| Comments: | 25 pages (10 pages main text + appendix), 9 figures, 20 tables |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.6; I.2.7 |
| Cite as: | arXiv:2610.35955 [cs.LG] |
| (or arXiv:2610.35955v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35955 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ning Yang [view email]
[v1]
Sun, 25 Oct 2026 03:02:21 UTC (249 KB)
来源：arXiv cs.AI · arxiv.org