arXiv cs.AI· Sheldon Liang, Ali Taghibakhshi, Saurav Muralidharan, Sharath Turuvekere Sreenivas, Ashwath Aithal, Pavlo Molchanov, Nima Tajbakhsh·· 12 小时前AI 评分38
AutoCompress：面向硬件约束的自动化模型压缩框架
AutoCompress: Automated Hardware-Guided Model Compression
AI 导读
AutoCompress 是一个硬件引导的压缩框架，将延迟、吞吐和显存约束直接融入可微分架构搜索，联合优化全局与逐层结构以生成异构架构。相比 Minitron，它在 RTX 5070、RTX 5080 和 H100 上八项基准平均准确率分别提升 6.57、2.35 和 0.57 个百分点。
正文 · 原文
Abstract:LLM pruning must preserve quality while meeting hardware- and workload-specific constraints on latency, throughput, and memory. Existing pruning workflows often target parameter counts, requiring practitioners to manually select model sizes that meet latency, throughput, and GPU memory limits. However, parameter count alone does not predict these metrics, which depend on the model architecture, workload, hardware, and serving engine. We introduce AutoCompress, a hardware-guided compression framework that directly incorporates such constraints into differentiable architecture search. AutoCompress jointly optimizes global and per-layer structure to produce heterogeneous architectures, guided by serving-engine cost models validated against end-to-end latency, throughput, and memory measurements. We evaluate AutoCompress on long-context local coding, batched datacenter serving, and latency-sensitive in-game interaction. Compared to the state-of-the-art Minitron pruning framework, AutoCompress improves average accuracy across eight benchmarks by 6.57, 2.35, and 0.57 percentage points on RTX 5070, RTX 5080, and H100, respectively. AutoCompress also compresses Nemotron 3.5 Super from 120B to 74B parameters for single-B200 BF16 serving at 128K context, retaining 95% of its mean benchmark score.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.08290 [cs.LG] |
| (or arXiv:2611.08290v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08290 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ali Taghibakhsh [view email]
[v1]
Thu, 5 Nov 2026 16:53:48 UTC (339 KB)
来源：arXiv cs.AI · arxiv.org