arXiv cs.AI· Seohyeon Cha, Huancheng Chen, Dongjun Kim, Haoran Zhang, Kevin Chan, Gustavo de Veciana, Haris Vikalo·· 10 小时前AI 评分22
CoreQ:面向 LLM 量化的失配校正逐次舍入方法
CoreQ: Mismatch-Corrected Successive Rounding for LLM Quantization
AI 导读
CoreQ 是一个无需训练的 PTQ 框架,通过对失配做几何分解,计算闭式、逐层自适应的校正系数,控制观测失配进入各层校准目标的比例,避免过拟合有限校准数据且无需调参。其有界束搜索扩展 K-CoreQ 以少量额外算力换取更好性能。在多个 LLM 系列、规模、位宽和量化设置下,CoreQ 与 K-CoreQ 的困惑度和下游准确率均优于强 PTQ 基线,代码已在 GitHub 开源。
正文 · 原文
Abstract:Post-training quantization (PTQ) enables efficient deployment of large language models by mapping pretrained weights to low-bit formats without retraining, typically using a small calibration set to minimize a layer-wise calibration objective. However, this sequential procedure induces a mismatch: errors from earlier quantized layers alter the inputs received by later layers, causing the activations to deviate from those of the full-precision model. Recent approaches introduce mismatch-aware calibration objectives to compensate for this effect, but leave open how much of the observed mismatch should shift each layer's calibration target. We show that fully applying the empirical mismatch correction can overfit finite calibration data, while using a fixed correction coefficient ignores variation in mismatch estimates across layers. To address these limitations, we propose CoreQ, a learning-free PTQ framework that uses a geometric decomposition of the mismatch to compute a closed-form, layer-adaptive correction coefficient. This coefficient controls how much of the observed mismatch enters each layer's calibration target, reducing overfitting to limited calibration data without coefficient tuning. CoreQ then applies efficient greedy successive rounding to the resulting triangular least-squares objective. Its bounded beam-search extension, K-CoreQ, trades modest additional compute for improved performance. Across multiple LLM families, scales, bit-widths, and quantization settings, CoreQ and K-CoreQ achieve lower perplexity and higher downstream accuracy than strong PTQ baselines. Code is available at GitHub.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.05902 [cs.LG] |
| (or arXiv:2602.05902v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.05902 arXiv-issued DOI via DataCite |
|
| Journal reference: | NeurIPS 2026 |
Submission history
From: Seohyeon Cha [view email]
[v1]
Thu, 5 Feb 2026 17:18:02 UTC (459 KB)
[v2]
Fri, 8 May 2026 19:18:18 UTC (993 KB)
[v3]
Wed, 18 Nov 2026 14:10:52 UTC (1,005 KB)
来源:arXiv cs.AI · arxiv.org