arXiv cs.AI· Haonan Dong, Chun-Wun Cheng, Angelica I. Aviles-Rivero·· 2 天前AI 评分31
Product Interactions：统一深度学习架构的代数形式化方法
Product Interaction: An Algebraic Formalism for Deep Learning Architectures
AI 导读
研究者提出 Product Interactions 代数形式化方法，将乘法而非线性变换作为架构的基本操作，统一了卷积、注意力、状态空间和等变模型。该框架将卷积和等变网络推导为对称约束的一阶乘积交互，把 Mamba 恢复为自交互阶从一增至三，并将注意力确立为三阶交互。基于此构建的简化代数动力学层在 CIFAR-10 上同时提升了准确率和吞吐量，优于 Mamba。
正文 · 原文
Abstract:Modern neural architectures are largely developed as distinct empirical constructions, despite recurring principles across convolutional, attention, state-space, and equivariant models. We introduce Product Interactions, an algebraic formalism that unifies these architectures under a common construction principle and turns their design into a systematic procedure. The core shift is representing data as algebraic elements and treating multiplication, rather than linear transformation alone, as a primitive architectural operation. Architectures are generated by composing multiplication operators, with input dependence organized by self-interaction order. Within this formalism, we derive convolutional and equivariant networks as symmetry-constrained first-order product interactions, recover Mamba from state-space models as an increase in self-interaction order from one to three, and establish attention as a third-order interaction from composing the same operator. Equivariance is incorporated by constraining the algebraic product via symmetry: translation equivariance yields convolution, while SO(3)-compatibility produces the Clebsch--Gordan product and tensor-field interactions. This perspective recasts architecture design as an algebraic construction problem: the choice of algebra defines admissible interactions, symmetry constrains their structure, and composition generates higher-order architectures. The framework is thus generative. Following these principles, we construct a simplified-algebra dynamical layer that improves both accuracy and throughput over Mamba on CIFAR-10. Experiments show that interaction order affects accuracy non-monotonically and symmetry constraints boost learnability. Product Interactions provides a mathematical methodology for analyzing existing architectures and systematically designing new ones.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.02573 [cs.LG] |
| (or arXiv:2602.02573v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.02573 arXiv-issued DOI via DataCite |
Submission history
From: Haonan Dong [view email]
[v1]
Sat, 31 Jan 2026 07:14:01 UTC (254 KB)
[v2]
Thu, 15 Oct 2026 11:28:46 UTC (1,938 KB)
来源：arXiv cs.AI · arxiv.org