跳到正文
arXiv cs.CL· Muhammad Junaid Ali, Smail Niar, El-Ghazali Talbi·· 10 小时前AI 评分22

面向延迟与模型大小优化的 LLM 多目标结构化剪枝框架

Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization

AI 导读

研究者提出一种硬件感知的多目标结构化剪枝框架,分两阶段同时优化 LLM 的推理延迟与模型大小。粗粒度阶段按深度剪除整个注意力与 MLP 块,细粒度阶段用并行贝叶斯优化(PBO)在延迟约束下搜索逐层剪枝比例。在 37.5% 和 50% 剪枝比例下,该方法在常识推理任务上优于现有方法,并显著降低推理成本。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large Language Models (LLMs) have achieved widespread adoption because of their strong reasoning and query-response capabilities. However, deploying them in embedded and edge computing environments remains challenging because of strict latency, memory, and energy constraints. Their large parameter counts and computational demands hinder efficient execution on resource-constrained platforms. Although model pruning has emerged as a viable solution for reducing scale while preserving performance, jointly optimizing layers, attention heads, and Multi-Layer Perceptron (MLP) dimensions remains highly complex. Exhaustively exploring this combined design space is computationally expensive and often leads to local optima or unstable configurations. To address these limitations, we propose a hardware-aware, multi-objective structured pruning framework. The proposed two-stage method explicitly targets latency and model size for efficient deployment on edge devices. In the coarse-grained stage, multi-objective depth pruning removes entire attention and MLP blocks to reduce computational load and memory usage. In the subsequent fine-grained stage, Parallel Bayesian Optimization (PBO) searches for the optimal layer-wise pruning ratios for pruning under latency constraints, while importance-based strategies rank the specific components to be pruned within each layer's allocated budget. Experimental results show that our approach reduces model complexity with minimal impact on commonsense reasoning tasks and zero-shot performance. Our method achieves a favorable trade-off among accuracy, latency, and model size, making it suitable for edge deployment. Across multiple LLMs at 37.5% and 50% pruning ratios, the proposed approach achieves better performance on commonsense reasoning tasks than existing methods while significantly reducing inference cost.
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as: arXiv:2607.22583 [cs.AI]
  (or arXiv:2607.22583v4 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2607.22583

arXiv-issued DOI via DataCite

Submission history

From: Muhammad Junaid Ali [view email]
[v1] Mon, 8 Jun 2026 09:38:13 UTC (4,945 KB)
[v2] Sun, 2 Aug 2026 07:11:59 UTC (6,532 KB)
[v3] Sat, 17 Oct 2026 15:24:34 UTC (5,173 KB)
[v4] Thu, 19 Nov 2026 11:31:48 UTC (5,173 KB)

来源:arXiv cs.CL · arxiv.org