arXiv cs.AI· Sergei Fedorchenko, Max Zehnder, Nikola Zubi\'c, Rong Zou, Davide Scaramuzza·· 14 小时前AI 评分24
Event-SAMA：基于神经形态事件流的语言引导视频目标分割
Event-SAMA: Language-Grounded Video Object Segmentation on Neuromorphic Event Streams
AI 导读
研究者提出 Event-SAMA，将 SAMA-4B 适配到事件流上，构建首个事件原生 RVOS 基线，在 MeViS valid_u 上取得 24.37 𝒥&ℱ，在转换后的 ReVOS valid 子集上取得 22.33 𝒥&ℱ。
正文 · 原文
Abstract:Event cameras offer microsecond temporal resolution, high dynamic range and no motion blur, but referring video object segmentation (RVOS) lacks event-stream datasets and models. We introduce Event-SAMA-239K, a synthetic-event conversion of the SAMA-239K grounded video-chat corpus and all additional SAMA training data: 30,650 video clips and 392,162 images. Optical-flow, FILM or saccade upsampling followed by a linear event generator preserves all mask, referring-expression and dialogue annotations. We also introduce Event-SAMA, adapting SAMA-4B with a two-channel polarity patch embedding initialised from pretrained RGB weights and a zero-initialised event adapter. End-to-end training yields 24.37 $\mathcal{J}\&\mathcal{F}$ on MeViS valid_u and 22.33 $\mathcal{J}\&\mathcal{F}$ on the converted ReVOS valid subset, establishing the first event-native RVOS baseline (RGB SAMA-4B: 55.4 on MeViS). Frozen RGB SAMA-4B with pseudo-RGB renderings or E2VID reconstructions of the same events scores 13.23 and 15.94 $\mathcal{J}\&\mathcal{F}$. Without retraining, Event-SAMA works on real event-camera recordings when the accumulation window matches their event density. A seed-controlled ablation with a measured noise floor of 0.47 $\mathcal{J}\&\mathcal{F}$ finds gains from diverse training sources over MeViS alone (+4.99), RGB-mean event-stem initialisation (+4.33), and a trainable SAM2 mask decoder (+2.76); the event adapter, partial ViT unfreezing and event polarity are within noise at this budget. An earlier, partially converted corpus snapshot trained with the identical recipe scores 1.38 $\mathcal{J}\&\mathcal{F}$ higher on MeViS and 1.12 higher on ReVOS. Both are single runs that also differ in schedule length, so we draw no conclusion about corpus scale.
| Comments: | 15 pages |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.39676 [cs.LG] |
| (or arXiv:2610.39676v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39676 arXiv-issued DOI via DataCite |
|
| Journal reference: | Conference on Neural Information Processing Systems Workshops (NeurIPSW), 2026 |
Submission history
From: Nikola Zubić [view email]
[v1]
Thu, 29 Oct 2026 09:22:49 UTC (4,665 KB)
来源：arXiv cs.AI · arxiv.org