arXiv cs.CL· Gilad D. Landau, Dulhan Jayalath, Oiwi Parker Jones·· 10 天前AI 评分31
Brain2Semantics2Text：用语义表示实现非侵入式语音解码
The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding
AI 导读
Brain2Semantics2Text 通过中间语义嵌入空间重建文本，将句子级 MEG 响应映射到语义流形后再反演为自然语言，无需词级对齐即可恢复高层语义。该方法针对非侵入式神经记录信噪比低、难以细粒度重建音素或单词的问题，利用跨皮层分布且时间尺度更慢的高层语义表示作为解码目标，与既有非侵入式 Brain2Text 方法相比提升了句子级结果。
正文 · 原文
Abstract:Non-invasive speech decoding remains constrained by the low signal-to-noise ratio of neural recordings, which makes fine-grained reconstruction of phonemes or individual words difficult. Motivated by neuroscientific evidence that high-level semantic representations are distributed across cortical regions and evolve over slower temporal scales, we hypothesize that semantic content may provide a more suitable target for non-invasive decoding than low-level acoustic or lexical features. We introduce Brain2Semantics2Text, a method that reconstructs text through an intermediate semantic embedding space. Our model maps sentence-level MEG responses into a semantic manifold and then inverts the predicted embeddings into natural language. This semantic bottleneck enables recovery of high-level meaning without word-level alignment. We describe the core principles of the approach, its implementation, and the strategies used to mitigate the challenges of learning a reliable neural-to-semantic mapping. Finally, we compare against prior non-invasive Brain2Text methods and show improved sentence-level results.
| Comments: | 12 pages, 8 figures |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2609.10296 [cs.CL] |
| (or arXiv:2609.10296v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.10296 arXiv-issued DOI via DataCite |
Submission history
From: Gilad Landau Mr [view email]
[v1]
Wed, 9 Sep 2026 15:11:57 UTC (4,313 KB)
[v2]
Wed, 7 Oct 2026 13:33:46 UTC (4,313 KB)
来源：arXiv cs.CL · arxiv.org