跳到正文
arXiv cs.CL· Shuai Wang, Malu Zhang, Mingquan Liu, Weihui Dai, Dehao Zhang, Jieyuan Zhang, Yimeng Shan, Zijian Zhou, Peng Zhou, Bolun Wang, Yang Yang·· 9 小时前AI 评分38

脉冲驱动视觉-语言-动作(VLA)模型:首个可端到端直接训练的 Spike-driven VLA 框架

Spike-driven Vision-Language-Action Model

AI 导读

研究者提出首个可端到端直接训练的 Spike-driven VLA 框架,用于机器人操作,包含脉冲视觉与指令编码器、Multi-Winner Spike Fusion 和 Spike Action Chunking Transformer 三个核心组件。在 LIBERO 和 Meta-World 上的实验显示,该模型以更少参数和更低估计推理能耗达到与传统 VLA 模型相当的性能。

正文 · 原文

Authors:Shuai Wang, Malu Zhang, Mingquan Liu, Weihui Dai, Dehao Zhang, Jieyuan Zhang, Yimeng Shan, Zijian Zhou, Peng Zhou, Bolun Wang, Yang Yang

View PDF HTML (experimental)

Abstract:Vision-language-action (VLA) models bridge multimodal understanding and robotic control, advancing the dominant paradigm for embodied intelligence. However, most existing models rely on large Transformers, whose latency and energy costs hinder deployment on resource-constrained platforms. Through sparse event-driven computation, spiking neural networks offer a promising paradigm for high-performance and energy-efficient computing. Here, we propose the first Spike-driven VLA framework enabling end-to-end direct training for robotic manipulation, which mainly comprises three core components. First, we develop spiking visual and instruction encoders for multimodal perception, encoding visual observations and language instructions into sparse, reliable spike representations for subsequent cross-modal fusion. Then, we introduce Multi-Winner Spike Fusion for instruction-guided scene understanding, using bidirectional top-$k$ winner-take-all spike routing to suppress background interference and yield fused memory. Finally, we propose a Spike Action Chunking Transformer that incorporates spiking cross-attention over the fused memory and the current robot state, enabling efficient end-to-end generation of continuous action chunks for robotic control. Extensive experiments on LIBERO and Meta-World demonstrate that Spike-driven VLA achieves competitive performance with fewer parameters and lower estimated inference energy than conventional VLA models. This work establishes a foundational framework for neuromorphic VLA modeling, paving the way for future advances in resource-efficient embodied intelligence.
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2609.39514 [cs.CL]
  (or arXiv:2609.39514v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2609.39514

arXiv-issued DOI via DataCite

Submission history

From: Shuai Wang [view email]
[v1] Wed, 30 Sep 2026 11:16:01 UTC (13,595 KB)
[v2] Tue, 17 Nov 2026 04:52:52 UTC (13,595 KB)

来源:arXiv cs.CL · arxiv.org