arXiv cs.AI· Lei Tian, Chunzheng Zhu, Kai Wu, Yifan Zhang, Haihua Yang·· 14 小时前AI 评分29
PACE：面向高效原生 MLLM 的帕累托自适应压缩框架
PACE: Pareto-Adaptive Compression for Efficient Native MLLMs
AI 导读
研究者提出 PACE，首个为采用 Multimodal-RoPE 的原生 MLLM 定制的视觉 token 压缩框架，在仅保留 25%-50% 视觉 token 时仍保持近乎无损性能，部分基准甚至提升，FLOPs 降低超 2 倍。
正文 · 原文
Abstract:Native multimodal large language models (native MLLMs) jointly encode visual and linguistic sequences within a unified autoregressive Transformer, emerging as a promising paradigm for multimodal understanding. Unlike conventional ViT-based MLLMs relying on external vision encoders, ViT-free native MLLMs tokenize pixel patches directly, potentially suffering severe visual token redundancy. A pilot study confirms this: unlike ViT-based models that degrade under compression, ViT-free MLLMs tolerate aggressive compression with near-lossless or even improved performance. Yet existing compression methods target ViT-based settings, leaving native MLLMs, which stand to gain the most, unexplored. To close this gap, we introduce an analysis pipeline centered on token discriminability and cross-dimensional heterogeneous redundancy, revealing that (i) redundancy is distributed independently and heterogeneously along height and width axes, and (ii) early-layer visual tokens are indiscriminable and uniformly redundant. Guided by these findings, we propose PACE (Pareto-Adaptive Compression for Efficient Native MLLMs), the first visual token compression framework tailored for native MLLMs with Multimodal-RoPE. Leveraging dimensional decoupling of positional encoding, PACE decomposes attention into two orthogonal spatial saliency estimates along height and width, gated by a positional attention signal, yielding direction-aware importance with negligible overhead. It further casts token budget allocation as Pareto optimization over performance preservation, inference efficiency, and spatial coverage, jointly identifying optimal compression depths and retention ratios. Experiments show PACE maintains near-lossless performance at only 25%-50% visual token retention, even improving several benchmarks, while delivering over 2x FLOPs reduction.
| Comments: | NeurIPS 2026 Accepted Paper |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.24005 [cs.CV] |
| (or arXiv:2610.24005v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.24005 arXiv-issued DOI via DataCite |
Submission history
From: Chunzheng Zhu [view email]
[v1]
Thu, 15 Oct 2026 17:58:25 UTC (26,662 KB)
[v2]
Thu, 29 Oct 2026 10:08:02 UTC (26,663 KB)
来源：arXiv cs.AI · arxiv.org