arXiv cs.AI· Vatsal, Soumil Malhotra, Priyanshu Chahal, Sanjeev Kumar·· 3 天前AI 评分22
Prune-and-Merge ViT：无需训练的 Vision Transformer token 压缩方法
Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression
AI 导读
研究者提出 Prune-and-Merge ViT，一种面向现成模型的免训练 token 压缩框架，结合基于类注意力的剪枝与基于相似度的合并来减少冗余并保留语义信息。在预训练的 ViT 和 DeiT 上，该方法在降低计算量的同时保持了鲁棒性，准确率优于现有 token 缩减方法。作者还引入 RAD 和 PFR 两项指标评估鲁棒性，分析显示混合 token 缩减不会引入类幻觉行为。
正文 · 原文
Abstract:Vision Transformers achieve strong performance but incur high computational costs from quadratic token interactions. Training-free token reduction methods improve efficiency, but their robustness remains underexplored. We propose Prune-and Merge ViT, a training-free framework for off-the-shelf models. It combines class attention-based pruning with similarity-based merging to reduce redundancy while preserving semantic information. To evaluate robustness we introduce two metrics. Relative Accuracy Drop (RAD) measures relative accuracy degradation under transformations, and Prediction Flip Rate (PFR) quantifies prediction consistency. On pretrained ViT and DeiT, Prune-and-Merge reduces computation competitively while maintaining robustness and better accuracy than existing token reduction methods. Our analysis suggests that hybrid token reduction does not introduce hallucination-like behavior and preserves prediction stability for common image transformations.
| Comments: | Accepted at the GlobalSouthAI Workshop at NeurIPS 2026; 12 pages |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18039 [cs.CV] |
| (or arXiv:2610.18039v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18039 arXiv-issued DOI via DataCite |
Submission history
From: Vatsal Singhal [view email]
[v1]
Tue, 13 Oct 2026 09:29:25 UTC (22 KB)
[v2]
Wed, 14 Oct 2026 04:42:32 UTC (22 KB)
来源：arXiv cs.AI · arxiv.org