arXiv cs.AI· Matthew McDermott, Florent Pollet, Payal Chandak, Gregory Kondas, Liat Friedman Antwarg, Noa Dagan, Isaac Kohane·· 1 天前AI 评分26
面向“任意任务推理”基础模型的理论框架：形式化定义与误差分解
A Theoretical Framework for Understanding and Building "Any-task inference" Capable Foundation Models
AI 导读
研究者提出“任意任务推理”（ATI）基础模型的形式化数学定义，将数据样本 x 与任务 t 直接映射到标签空间，并引入“任务兴趣空间”概念。该框架证明所有 ATI 模型可经“通用任务归约”分解为编码器、模型与解码器三部分，进而把模型误差拆分为设计项与已实现模型超额项。作者据此给出生成式与判别式两类 ATI 实现原型，并用 5 个真实基础模型案例验证定义的适用性。
正文 · 原文
Abstract:In this work, we provide a formal framework and mathematical definition for the subset of foundation models focused on enabling the property of "any-task inference" (ATI) (similar to zero-shot inference), rather than those designed for pre-training fine-tuning. To achieve this mathematical definition, we formalize the notion of a "space of tasks of interest," which we include in the problem domain, then provide an interface-based formal definition for ATI foundation models as those that map samples of data $x$ and tasks $t$ to predictions over the space of labels directly. While simple, this definition allows us to show that all ATI foundation models can be decomposed through a "universal task reduction" lens into encoder, model, and decoder aspects. We can leverage this decomposition to also decompose foundation model error into terms corresponding to different aspects of this reduction, stratified into a collection of "design terms," which are theoretical properties that can inform one of whether a particular proposed ATI foundation model can achieve optimality or not due to the structure of the proposed task reduction, and "realized model excess" terms which capture interactions between learned and programmatic components of foundation model systems. Lastly, we can use this perspective to offer two "prototypical" recipes for realizing ATI foundation models in diverse domains, which we term generative and discriminative variants of ATI foundation models. This exercise is grounded throughout by 5 real-world examples of foundation models in different settings, varying greatly in how these capabilities are realized, showing the applicability of this definition to real-world settings.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.40934 [cs.LG] |
| (or arXiv:2610.40934v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40934 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Florent Pollet [view email]
[v1]
Thu, 29 Oct 2026 20:08:35 UTC (79 KB)
来源：arXiv cs.AI · arxiv.org