arXiv cs.CL· Mingyang Song, Mao Zheng·· 11 小时前AI 评分31
重新审视 LLM 剪枝：从测试时扩展视角评估剪枝模型
Rethinking LLM Pruning via Test-Time Scaling
AI 导读
研究通过匹配采样预算下的 pass@k 对比原始与剪枝模型，发现部分模型在 expert、FFN 和 layer 剪枝后虽 pass@1 下降，推理覆盖范围仍可持平甚至超过原模型，并能解出原模型未解的问题。但严重剪枝下正确解更难采样，重复增多、生成可靠性下降，总 token 消耗可能大幅上升。因此剪枝应同时以可解问题范围和获取正确解的采样成本两项指标评估。
正文 · 原文
Abstract:Structured pruning reduces the size of large language models, often at the cost of lower accuracy. Whether this decline reflects diminished reasoning capacity or greater difficulty in sampling correct solutions is less clear. We examine this question across expert, FFN and layer pruning on four mathematical and coding benchmarks, comparing original and pruned models through pass@k at matched sampling budgets. Even after substantial pruning, some models match or exceed the original model's reasoning coverage despite lower pass@1, and solve problems left unsolved in the original samples. Retaining broad coverage, however, does not mean that correct solutions remain easy to obtain. Under severe pruning, diverse generated chain-of-thought (CoT) traces can coexist with increased repetition and less reliable generation. As correct answers become harder to sample, the additional attempts can sharply increase total token use even when individual responses become shorter. This separation between coverage and sampling efficiency also clarifies expert-selection gains, as methods with markedly different pass@1 can solve the same problems at large budgets yet require different numbers of attempts to reach a common coverage target. Pruning should therefore be assessed by both the range of problems a model can still solve and the sampling cost of obtaining correct solutions.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.01133 [cs.CL] |
| (or arXiv:2611.01133v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.01133 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mingyang Song [view email]
[v1]
Sat, 31 Oct 2026 17:22:59 UTC (1,210 KB)
来源：arXiv cs.CL · arxiv.org