arXiv cs.CL· Michael A. Lepori, Laila Johnston, Aalok Sathe, Zhuonan Yang, Roman Feiman, Ellie Pavlick·· 12 小时前AI 评分26
语言模型少样本概念归纳的算法解释：Amortized Direction Accentuation（ADA）
An Algorithmic Account of Few-Shot Concept Induction in Language Models
AI 导读
研究提出 Amortized Direction Accentuation（ADA），一种解释语言模型如何在少样本概念归纳中判定特征相关性的算法假设。ADA 认为概念归纳通过在有摊销知识塑造的特征空间中形成示例的摘要表示来实现，并在语言模型的少样本概念归纳任务上得到验证。该工作为语言模型的上下文机制提供了新视角。
正文
Abstract:Both models and minds not only encode features that describe an entity, but can specifically represent which features of that entity are relevant for a given context. This core capability underlies in-context learning in language models and a variety of cognitive phenomena in minds, yet it is unclear how intelligent systems identify and promote relevant features. In this work, we propose Amortized Direction Accentuation (ADA), an algorithmic hypothesis that provides an explanation for how relevance is determined in the context of few-shot concept induction --- the ability to infer a latent concept from a small number of positive examples. Under ADA, concept induction proceeds by forming a summary representation of the examples in a feature space that is shaped by amortized knowledge about which features tend to be useful for finding commonalities between items. We investigate and validate ADA in language models using a few-shot concept induction task. This work provides a novel perspective on the mechanisms supporting contextualization in language models and a promising hypothesis for a core aspect of cognition.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.35533 [cs.CL] |
| (or arXiv:2610.35533v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35533 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Michael Lepori Jr. [view email]
[v1]
Mon, 26 Oct 2026 14:48:07 UTC (1,000 KB)
来源：arXiv cs.CL · arxiv.org