arXiv cs.AI· Jianyu Zhang, L\'eon Bottou·· 2 天前AI 评分32
开放世界 AI 的三项要求：一项基于 8B 语言模型的实证研究
Open-world AI: a Tale of Three Requirements
AI 导读
研究提出开放世界 AI 需满足三项要求：推理时学习、丰富表征与预测性解耦，并将任务建模为共享潜在因子、分解训练目标推导得出。在最高 8B 的语言模型上验证显示，满足要求的模型会随相关任务数据积累而提升，其余模型则退化，且扩大规模或层类型交错都无法修复。分析还预测了一致内部学习者的必要性以及跨相关任务长上下文的收益。
正文 · 原文
Abstract:The closed world asks AI for specialized competence and lets it manufacture the iid data its theory assumes: a fixed criterion licenses collecting examples until the task is covered. The open world asks for versatility --- taking up tasks a machine was not built for, from few examples and with little prior from a human designer --- and offers data that is grouped, not shuffled. Its tasks are not known in advance, so a machine must do inference-time learning (1st requirement). Modeling tasks as sharing latent factors and decomposing the training objective, we derive two more: a *rich representation*, retaining more than iid generalization requires, and a predictive disentanglement that aligns inputs and outputs into matched low-dimensional groups. The analysis further predicts the need for a consistent inner learner, and the benefit of long contexts spanning related tasks. We test these claims on language models up to 8B, asking whether a model has a property (qualitative) rather than how much it wins (quantitative). Models that meet the requirements improve as data from related tasks accumulates; the rest degrade, and neither scale nor layer-type interleaving repairs this.
| Comments: | Code: this https URL |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.20782 [cs.AI] |
| (or arXiv:2610.20782v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20782 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jianyu Zhang [view email]
[v1]
Wed, 14 Oct 2026 18:28:16 UTC (1,791 KB)
来源：arXiv cs.AI · arxiv.org