arXiv cs.AI· Zhen Liu, Zhiqin Yang, Yonggang Zhang, Jianyi Liu, Jinjun Wang, Jingwen Fu·· 1 天前AI 评分29
AutoQS：用大语言模型自动设计量化策略
AutoQS: Automated Quantization Strategy Design with Large Language Models
AI 导读
AutoQS 利用 LLM 在有限评估预算内自动搜索逐层与逐算子的 group-size 分配，在固定权重精度下兼顾模型质量与存储成本。在 Qwen2.5-7B-Instruct 上以四位权重量化时，其七任务平均准确率分别比 uniform-group GPTQ 和 AWQ 高 0.70 和 1.12 个百分点。方法结合廉价质量代理评估、候选仓库与周期性实际量化反馈来指导搜索。
正文 · 原文
Abstract:Quantization reduces LLM deployment costs, but effective strategies must balance model quality and storage. LLM-driven search can automate strategy design through repeated proposal and evaluation. This process must identify useful strategies within a limited evaluation budget. The key challenge is that \textit{actual quantization is expensive, while inexpensive proxies may not reflect quantized-model performance.} To address this challenge, we present AutoQS, an \textbf{Auto}mated \textbf{Q}uantization \textbf{S}trategy Design with LLMs. AutoQS keeps weight precision fixed and searches group-size allocations across layers and operators under deployment constraints. Evaluation history guides coordinated LLM proposals, while deterministic validation ensures compatible allocations. An inexpensive quality proxy evaluates candidates, and an candidate repository retains alternative allocations to inform subsequent proposals. Periodic actual quantization complements these proxy evaluations with measured quality and checkpoint storage, providing feedback for further search within the evaluation budget. Experiments on Qwen2.5-7B-Instruct show that AutoQS improves seven-task average accuracy over uniform-group GPTQ and AWQ by 0.70 and 1.12 percentage points, respectively, at four-bit weight precision.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.11975 [cs.AI] |
| (or arXiv:2611.11975v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.11975 arXiv-issued DOI via DataCite |
Submission history
From: Zhen Liu [view email]
[v1]
Mon, 9 Nov 2026 06:59:18 UTC (185 KB)
[v2]
Wed, 11 Nov 2026 05:23:20 UTC (185 KB)
来源：arXiv cs.AI · arxiv.org