arXiv cs.CL· Oladimeji Anthonio, Dimeji Olawuyi, Oloruntoba Ajayi, Temilola Aderemi·· 12 小时前AI 评分38
五个架构共享一个空间：无需权重合并的跨模型表征对齐
Five Architectures, One Shared Space Cross-Model Representational Alignment Without Weight Merging
AI 导读
五个冻结且独立训练的模型（四个来自不同机构的 Transformer 和一个状态空间模型）仅凭激活值即可建立统计稳健的表征对应，最强出现在各自最后一层，状态空间模型最弱配对仍达随机水平的 35-43 倍。
正文 · 原文
Abstract:Combining language models is usually done by averaging weights, which works only when the models share a common starting point. We ask whether five frozen, independently trained models with different internal designs (four transformers from four organizations and one state-space model) can be related through their activations alone, with no weight merging. Using permutation tests, an independent replication probe set, depth sweeps, pooling ablations and confound checks, we find statistically robust correspondence between how the five models organize the same text. It is strongest at each model's final layer and weaker for the state-space model, whose weakest pairing is still 35-43 times chance. We use this to build a shared space with Generalized Procrustes Analysis, in which no model is a fixed reference. At the final layer, mean held-out retrieval accuracy over the twenty ordered model pairs is 0.91 to 0.93 on two probe sets (chance 0.03), beating pairwise linear maps (0.82 to 0.85), but the space is worse than those maps in the middle layers. It transfers to ordinary English of unfamiliar grammatical form and fails on mathematical and logical notation. We then test whether the space supports a router that picks among the five models on gradeable tasks. Across four task types we find no case where routing beats always using the strongest model, and we report this as a negative result. A correctness signal that looked strong turned out to be category membership in disguise. All experiments ran on one laptop without a GPU.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.33495 [cs.CL] |
| (or arXiv:2610.33495v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33495 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Oladimeji Anthonio [view email]
[v1]
Sat, 24 Oct 2026 06:52:49 UTC (110 KB)
来源：arXiv cs.CL · arxiv.org