arXiv cs.AI· Serwan Jassim, Victoria Bosch, Daniel Anthes, Elia Bruni, Tim Kietzmann·· 14 小时前AI 评分34
JEPA 自监督学习在感官诱发 fMRI 中产生跨被试可对齐表征
Latent Prediction Yields Cross-Subject Alignable Representations in Sensory-Evoked fMRI
AI 导读
研究者将联合嵌入预测架构（JEPA）适配为仅用脑数据训练的被试专属编码器，编码器从未接触刺激图像或其衍生表征。在 Natural Scenes Dataset（NSD）上，该自监督方法在脑描述生成与图像重建任务上达到与使用图像和语言数据端到端训练的富参数监督流程相当的性能。
正文 · 原文
Abstract:Decoding sensory-evoked functional magnetic resonance imaging (fMRI) data is a central analytical tool in cognitive neuroscience. Here we ask whether self-supervised learning (SSL), which has been scarcely explored in this domain, can yield representations that support strong downstream performance. To do so, we adapt a joint embedding predictive architecture (JEPA) to train participant-specific encoders on brain data alone. The encoders never see the stimulus images or any representation derived from them. Applied to the Natural Scenes Dataset (NSD), we evaluate our learned brain embeddings on the downstream tasks of brain captioning and image reconstruction, as well as cross-participant generalization. Our self-supervised approach achieves competitive performance with parameter-rich supervised pipelines that are trained end-to-end with access to image and language data. We further find that encoders trained independently for different participants can be aligned fully unsupervised, using neither images nor stimuli shared between them. After alignment, a neural decoding head trained on one participant can be applied to another without fine-tuning and retains most of its within-subject performance on image reconstruction and brain captioning. By removing the need for images and shared stimuli, our approach is a step towards pooling fMRI data across participants and, eventually, across datasets.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neurons and Cognition (q-bio.NC) |
| Cite as: | arXiv:2610.40280 [cs.LG] |
| (or arXiv:2610.40280v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40280 arXiv-issued DOI via DataCite |
Submission history
From: Serwan Jassim [view email]
[v1]
Thu, 29 Oct 2026 15:33:41 UTC (20,289 KB)
来源：arXiv cs.AI · arxiv.org