arXiv cs.AI· Luis F. Rosario Freytes·· 3 天前AI 评分15
PreNorm Transformer 中训练依赖的计算语义：架构先于公式
Architecture Before the Formula: Training-Dependent Computational Semantics in Transformers
AI 导读
研究提出"深度学习外在论"（DLE），认为运算的计算角色不仅取决于其输入输出函数，还取决于产生输入和接收输出的过程。作者在维度标记的 PreNorm Transformer 中证明，两个单射生产者上下文可保留全部前驱区分却改变固定局部与注意力接收器族是否等价，并推导出梯度下降下协方差相关的几何衰减与 Q/K-softmax 训练下的 $(t^{-1/2})$ 衰减。
正文 · 原文
Abstract:An operation's input-output function does not, by itself, determine its computational role within a represented process. That role also depends on the processes producing its inputs and receiving its outputs, a philosophical stance termed deep learning externalism (DLE). We develop a mathematical account of represented computation that distinguishes functional equivalence, retained predecessor distinctions, restricted receiver capability, and context-dependent effective architectures. The resulting equivalences depend on which computational relations a scientific description must preserve. We prove that two injective producer contexts can preserve every predecessor distinction while changing whether fixed local and attention receiver families are equivalent.
We investigate these relations in a dimension-labelled PreNorm Transformer and their evolution under training. Residual bypasses restrict complete-state information identifications to two merges per block, while finer branches can change receiver accessibility. Exact gradient-descent models yield covariance-dependent geometric attenuation under learned producer routing and algebraic $(t^{-1/2})$ attenuation under Q/K-softmax training, with optimal bounded-receiver recovery errors derived in both cases. Coupled producer-attention dynamics establish how learning jointly reorganizes the presented features and receiver allocation. Finally, present functional or receiver-profile equivalence need not determine future behavior under an optimizer. In an input-conditioned attention model, two accessibility measurements form an exact reduced predictive state, whereas one alone does not.
| Comments: | 117 Pages |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2601.11618 [cs.LG] |
| (or arXiv:2601.11618v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2601.11618 arXiv-issued DOI via DataCite |
Submission history
From: Luis Rosario Freytes [view email]
[v1]
Sat, 10 Jan 2026 13:43:01 UTC (62 KB)
[v2]
Wed, 22 Jul 2026 02:16:17 UTC (1 KB) (withdrawn)
[v3]
Wed, 12 Aug 2026 21:16:43 UTC (32 KB)
[v4]
Sun, 11 Oct 2026 01:55:01 UTC (373 KB)
来源：arXiv cs.AI · arxiv.org