arXiv cs.CL· Yujian Liu, Li An, Zelin He, Jacob Helwig, Jiabao Ji, Rohit Jain, Siyu Zhu, Shiyu Chang·· 13 小时前AI 评分52
构建 AI 研究团队自主发现线性注意力架构
Building AI Research Teams for Autonomous Discovery of Linear Attention Architectures
AI 导读
研究团队构建了一支由不同学科背景的持久化 AI 研究员组成的团队，成员各自保留私有研究历史、共享实验证据，并通过自适应分配获得研究机会，用于搜索线性注意力架构。在 1.3B 规模的语言建模、常识推理以及合成与真实检索任务上，该团队发现的架构优于 GDN2。对照实验显示，去掉学科背景会让搜索过早停滞，去掉私有历史会在相同共享证据下进展更少，统一选择策略在同等预算下不如自适应分配。
正文 · 原文
Abstract:Autonomous agents can increasingly propose ideas, modify code, and run experiments, creating an opportunity to accelerate scientific discovery using teams of collaborative AI researchers. To escape local minima in the discovery, we propose to encourage orthogonal exploration by constructing the team using agents with varying expertise. However, realizing this design requires more than diverse backgrounds. When experiments are costly, a team can test only a limited number of ideas, thereby creating the need to balance exploration and exploitation when selecting a subset of researchers to propose and execute new experiments. Furthermore, each result should shape what the team tries next. We therefore ask how AI researchers with different scientific backgrounds can build on one another's findings and sustain progress under a limited experimental budget. To address these challenges, we build a team of persistent AI researchers, each grounded in a different discipline, who keep private research histories, share experimental evidence, and receive research opportunities through adaptive allocation. We demonstrate the effectiveness of this design by deploying the team to search for linear attention architectures, where it outperforms GDN2 at 1.3B scale on language modeling, common-sense reasoning, and synthetic and real-world retrieval tasks. In controlled comparisons, removing any single organizational component from our design yields markedly weaker discoveries: without disciplinary backgrounds, the search plateaus early; without private histories, researchers make less progress despite identical shared evidence; and uniform selection falls short of adaptive allocation within the same budget. These results suggest that sustained autonomous discovery depends not only on stronger agents, but also on who explores, what they remember, and how research effort is allocated.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.28887 [cs.CL] |
| (or arXiv:2610.28887v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28887 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yujian Liu [view email]
[v1]
Tue, 20 Oct 2026 20:09:31 UTC (10,233 KB)
来源：arXiv cs.CL · arxiv.org