arXiv cs.CL· Jie Wang, Tong Zhu, Jiahao Kuang, Zhe Xu, Mowen Ruan, Yunkai Chai, Yuliang Liu, Dehong Zheng, Rubin Wei, Yu Cheng, Bowen Zhou, Tao Ji, Zhouhan Lin, Qi Zhang·· 2 天前AI 评分38
ArchSpace:记录、评审与引用 LLM 架构研究的公开平台
ArchSpace: Recording, Reviewing, and Referencing LLM Architecture Research
AI 导读
ArchSpace 是一个面向 LLM 架构研究的公开平台,用于记录每项架构提案的实验与推理过程、接受社区评审并被后续工作引用。平台利用 scaling laws 从较小规模实验预测更大规模的模型 loss,以在昂贵的大规模训练前评估想法。初始参考资源包括 Qwen3.5 和 OLMo3 基线、拟合的 scaling laws 以及可复用的训练配方。
正文 · 原文
Authors:Jie Wang, Tong Zhu, Jiahao Kuang, Zhe Xu, Mowen Ruan, Yunkai Chai, Yuliang Liu, Dehong Zheng, Rubin Wei, Yu Cheng, Bowen Zhou, Tao Ji, Zhouhan Lin, Qi Zhang
Abstract:Model architecture shapes the scaling behavior, training efficiency, and inference cost of LLMs, making it a central focus of both industry and academia. In industry, teams such as those behind DeepSeek, Qwen, and Kimi routinely introduce architectural innovations, yet public reports rarely document which ideas succeeded, which were abandoned, and why. In academia, limited compute and inconsistent baselines often restrict evaluations to small-scale, single-point comparisons, making it difficult to establish whether improvements hold at larger scales or translate into broader impact. ArchSpace addresses both gaps as a public platform for Recording, Reviewing, and Referencing Architecture Research: each proposal is recorded with its experiments and reasoning, reviewed by the community, and referenced by later work. To evaluate ideas before costly large-scale training, ArchSpace leverages scaling laws to predict model loss at larger scales from smaller experiments, a practice widely adopted in industry. We provide initial references that make the platform usable: Qwen3.5 and OLMo3 baselines, fitted scaling laws, and a reusable training recipe. Ideas validated through this review can in turn become new baselines, and ArchSpace thereby aims to broaden the range of ideas that shape the next generation of models.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.17332 [cs.CL] |
| (or arXiv:2611.17332v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17332 arXiv-issued DOI via DataCite |
Submission history
From: Jie Wang [view email]
[v1]
Thu, 12 Nov 2026 18:59:52 UTC (1,427 KB)
[v2]
Fri, 13 Nov 2026 18:58:31 UTC (1,397 KB)
来源:arXiv cs.CL · arxiv.org