BrainBind：将同步 EEG 与 fMRI 映射到长视频表征空间

View PDF HTML (experimental)

Abstract:Real-world vision is dynamic and scene-rich, while EEG and fMRI resolve it at complementary temporal and spatial scales. Most neural decoding systems, however, align one modality in isolation and mostly against static images or short clips. We study the synchronized case: EEG and fMRI are recorded during the same long-form naturalistic viewing, giving a shared stimulus but also recurring narrative context and many semantically similar candidates. We present BrainBind, a framework that maps synchronized EEG and fMRI into frozen long-form video embedding spaces and treats those spaces as common visual-semantic references, not literal neural codes. On CineBrain, a five-seed Recall@K 2 x 4 matrix across two video anchors and four system configurations shows distinct retrieval profiles. With Omni, fMRI organizes the full candidate list most strongly (MRR = 23.82, nDCG@10 = 25.31), while Joint-Dual gives the highest mean label-aware R@1 (13.52%) with a distinct rank-wide profile. The complementary episode-disjoint evaluation quantifies the contribution of repeated stimulus identity and narrative context to the random split. Across these experiments, proxy choice and system configuration produce distinct top-rank and rank-wide retrieval profiles within the present synchronized dataset. The code is available at this https URL.

Comments:	Accepted at BMVC 2026. 26 pages, 4 figures, including supplementary material. Code: this https URL
Subjects:	Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
Cite as:	arXiv:2611.05581 [cs.AI]
	(or arXiv:2611.05581v1 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2611.05581 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Shixiao Wang [view email]
[v1] Tue, 3 Nov 2026 20:11:50 UTC (10,791 KB)