arXiv cs.AI· Aditya Gupta, Jean Sebastien Dandurand, Kai Qiu, Rohan Choudhury, Laszlo A. Jeni·· 11 小时前AI 评分31
SPLIT：为 Vision Transformer 学习分层 patch 拆分策略以减少 token 数
Learning Hierarchical Patch Splitting Policies for Faster Vision Transformers
AI 导读
研究者提出 SPLIT（Split Policy Learned Image Tokenization），让 Vision Transformer 依据模型认为有意义的内容而非像素级熵等启发式指标来分配 token 分辨率，生成更短的 token 序列。
正文 · 原文
Abstract:Vision Transformers (ViTs) are bottlenecked by the length of their input token sequence. Standard patchification fixes this length with a single patch size applied across the image, regardless of regional content. Prior work on adaptive patch sizing reduces token count by assigning coarser patches to visually simple regions, but relies on noisy pixel-level heuristics such as local entropy to make patchification decisions. We argue that this compute allocation should be grounded in the model's interpretation of task relevant information, not in raw pixel statistics. We introduce Split Policy Learned Image Tokenization which learns to allocate token resolution based on what the model finds meaningful, producing shorter token sequences that are better matched to task-relevant content. SPLIT matches the accuracy of full-resolution ViT classification baselines while using 30% fewer tokens. The same gains hold on dense prediction: SPLIT achieves competitive accuracy on COCO and ADE20K, demonstrating that semantically aligned patchification generalises well beyond classification.
| Comments: | Accepted at NeurIPS 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.05742 [cs.CV] |
| (or arXiv:2611.05742v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05742 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aditya Gupta [view email]
[v1]
Tue, 3 Nov 2026 23:28:16 UTC (21,119 KB)
来源：arXiv cs.AI · arxiv.org