arXiv cs.AI· Saman Rahbar·· 9 天前AI 评分23
相似度分数不等于结构：大脑对齐与跨语言迁移的可靠性问题
The Score Is Not the Structure: Brain Alignment and Cross-Lingual Transfer
AI 导读
一项 NeurIPS 2026 LP4FM 接收研究指出，相似度分数未必反映模型与大脑或跨语言共享的真实结构。跨 17 种语言测试中，语法性探针在 4 种语言上仅达随机水平，且距离越远迁移越差（r = -0.66）；将 272 个语言对视为独立时转向效应 p = 0.0006，而以 17 种语言为单位时 p = 0.155。
正文 · 原文
Abstract:Similarity scores are often offered as evidence that a model shares structure with the brain or across languages. We ask what such a score reads when that structure is removed, or when the instrument measuring it does not work, in two settings. Across seventeen languages, a grammaticality probe transfers worse between more distant languages (r = -0.66). But the probe's own accuracy falls along the same axis and is at chance in four languages, four of the five most distant. Dropping them halves the explained variance, and because it also narrows the range of distances, the design cannot say how much of the gradient is the instrument. Counting the 272 language pairs as independent gives p = 0.0006 for a steering effect that is null when the seventeen languages are the unit (p = 0.155). In brain alignment, a training objective raises a language model's similarity to fMRI responses from 0.10 to 0.34 (reliability ceiling 0.54), yet targets with the brain correspondence destroyed still reach 0.31. A shuffled target scores about k/n against the real one, for rank k and n sentences, a baseline that can be computed before any model is trained. Both interventions work, yet we detect no distance-graded steering effect and no syntactic benefit from alignment. Before reading a correspondence score, measure how much of it survives without the correspondence.
| Comments: | 12 pages, 2 figures, 1 table. Accepted as a poster at the NeurIPS 2026 - Linguistic Principles for Foundation Models (LP4FM). Code: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neurons and Cognition (q-bio.NC) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2610.03827 [cs.CL] |
| (or arXiv:2610.03827v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03827 arXiv-issued DOI via DataCite |
Submission history
From: Saman Rahbar [view email]
[v1]
Fri, 2 Oct 2026 09:21:49 UTC (67 KB)
[v2]
Wed, 7 Oct 2026 19:00:14 UTC (67 KB)
来源：arXiv cs.AI · arxiv.org