arXiv cs.CL· Zehui Feng, Weichuan Wang, Xiaohan Chen, Cuntai Guan, Ting Han·· 12 小时前AI 评分37
语音脑机接口需要更大模型吗？BEST 框架用 3.85% 参数实现更低 WER
Do Speech BCIs Need Larger Models? Rethinking Neural Decoding beyond Scaling
AI 导读
BEST（Brain-Encoding-Speech-Text）两阶段框架在 Brain-to-Text '24 和 '25 基准上分别取得 6.19% 和 1.62% 的词错误率，仅需同类方法 3.85% 的参数量。该框架用分层中间表示和紧凑语音解码器直接将神经活动映射为文本，避免大型生成模型与音素中心流水线。研究还指出解码器扩展收益非单调、跨模态对齐比模型规模更关键、过拟合会损害序列级性能。
正文 · 原文
Abstract:Recent speech brain-computer interfaces (BCIs) increasingly rely on large models and complex multi-stage pipelines for neural decoding. However, in speech BCI, where training data are scarce, noisy, and expensive to collect, the benefits of model scaling remain unclear. This work revisits a fundamental question: are large models truly necessary for accurate neural decoding? We propose BEST (Brain-Encoding-Speech-Text), a two-stage framework for efficient neural-to-text decoding. Instead of relying on large generative models or phoneme-centric pipelines, BEST introduces hierarchical intermediate representations and a compact speech-oriented decoder that directly maps neural activity to text. This design preserves essential structure while significantly reducing computational cost, enabling robust generalization in low-resource settings and practical deployment under constrained resources. On the Brain-to-Text '24 and '25 benchmarks, BEST achieves word error rates (WERs) of 6.19% and 1.62%, respectively, using only 3.85% of the parameters required by comparable methods. Beyond these results, we provide three key insights: (1) decoder scaling yields non-monotonic gains; (2) cross-modal alignment is more critical than model size; and (3) overfitting degards sequence-level performance. Representation analysis further reveals a consistent neural-audio-text alignment pathway.
| Comments: | 25 pages, Accepted by NeurIPS 2026 as a Main Track conference paper |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.23175 [cs.CL] |
| (or arXiv:2610.23175v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23175 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zehui Feng [view email]
[v1]
Fri, 16 Oct 2026 09:12:58 UTC (8,020 KB)
来源：arXiv cs.CL · arxiv.org