arXiv cs.AI· Zecheng Hao, Zihan Huang, Shenghao Xie, Jiyuan Zhang, Zhaofei Yu, Tiejun Huang·· 10 小时前AI 评分24
AdaS-ViT:面向脉冲视觉 Transformer 的超高能效测试时自适应路由
AdaS-ViT: Test-Time Adaptive Routing for Spiking Vision Transformers with Ultra-High Energy Efficiency
AI 导读
AdaS-ViT 是一种面向脉冲视觉 Transformer 的测试时自适应路由架构,通过 AdaE-IF 依据每个脉冲 token 的注意力分数,将显著区域路由到细粒度脉冲模式、其余区域采用低能耗粗粒度模式。
正文 · 原文
Abstract:Spiking Neural Networks (SNNs) promise event-driven and energy-efficient vision, yet existing spiking vision transformers (S-ViTs) largely confine spiking tokens to fixed and uniform firing schemes along the temporal, spatial and channel dimensions, whereas suppressing the computational redundancy thus incurred calls for adaptive firing mechanisms that operate jointly across all three axes at test time. To this end, we propose the Adaptive Efficient Integrate-and-Fire (AdaE-IF) model, which consults the attention score of each spiking token and routes salient regions to fine-grained firing patterns while assigning energy-frugal coarse patterns to the remaining regions. Building on AdaE-IF, we further introduce Spiking Gated Self-Attention (S-GSA) and Spiking Local-Gated Unit (S-LGU), which enforce selective spike emission at token and channel granularity. We integrate the aforementioned mechanisms and modules into AdaS-ViT, a test-time adaptive-routing architecture trained with a progressive spatio-temporal curriculum that stabilizes the co-adaptation of firing precision and routing masks. Extensive experiments across benchmarks spanning diverse data scales and modalities demonstrate that AdaS-ViT consistently achieves highly competitive performance. Notably, on ImageNet-1k it consumes only 1.02 to 2.45 millijoules per image and attains an accuracy-per-energy ratio of 32$\times$ to 72$\times$, operating in an economical inference regime and thereby pushing the energy-efficiency boundary further.
| Subjects: | Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2611.23728 [cs.NE] |
| (or arXiv:2611.23728v1 [cs.NE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23728 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zecheng Hao [view email]
[v1]
Wed, 18 Nov 2026 02:40:45 UTC (462 KB)
来源:arXiv cs.AI · arxiv.org