ASI-Arch 通过自主进化发现 105 个线性注意力架构

Neural Architecture Discovery via Autonomous Evolution

AI 导读

论文提出 ASI-Arch，一个通过研究、实验、分析、更新闭环自主开展神经网络架构研究的系统。在线性注意力任务上，它运行了 1773 次迭代实验，发现 105 个 SOTA 架构，其中最佳架构相对 DeltaNet 的提升接近 Mamba2 所带来增益的三倍。论文还分析了框架各部分在这一高难度研究场景中的贡献，该工作已被 COLM 2026 接收。

正文 · 原文

View PDF HTML (experimental)

Abstract:Recent progress in LLM agents has advanced the prospect of autonomous research. Yet whether AI can complete difficult long-horizon tasks, especially those that advance AI research itself, remains largely unexplored. We present ASI-Arch, a system for AI-driven AI research that autonomously conducts neural architecture research through a closed-loop research-experiment-analyze-update process. Applied to linear attention, ASI-Arch ran 1,773 iterative experiments and discovered 105 state-of-the-art architectures. Its best architecture improves over DeltaNet by nearly three times the gain achieved by Mamba2. Beyond the final performance gains, we analyze the contributions of different parts of the framework in this hard research setting, shedding light on what enables autonomous progress in complex AI research tasks.

Comments:	44 pages, 6 figures. Accepted at COLM 2026. Camera-ready version
Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2507.18074 [cs.AI]
	(or arXiv:2507.18074v2 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2507.18074 arXiv-issued DOI via DataCite

Submission history

From: Weixian Xu [view email]
[v1] Thu, 24 Jul 2025 03:57:27 UTC (2,130 KB)
[v2] Thu, 8 Oct 2026 09:05:16 UTC (582 KB)

来源：arXiv cs.AI · arxiv.org