斯坦福论文《A Circuit for In-Context Learning》揭示小模型上下文学习电路
斯坦福论文揭示上下文学习电路：小模型靠重复标签触发查找而非学会新模式
斯坦福论文《A Circuit for In-Context Learning》（arXiv:2611.05344）发现，小语言模型的 few-shot 学习并非从示例中学到新规律，而是通过标签查找已有映射，且仅在标签重复时才触发这一查找。
Rohan Paul· @rohanpaul_ai · X·2026-11-05 10:22· 38 分钟前AI 评分51
AI 导读
Stanford 论文《A Circuit for In-Context Learning》（arXiv:2611.05344）发现小语言模型的 few-shot 学习不是从示例学到新规律，而是通过标签查找已有映射，仅在标签重复时触发。
正文 · 原文
New Stanford paper finds that showing a small AI language model more examples cannot teach it a pattern it does not already notice, such as the third letter of a word.
finds that LLMs learn from few-shot examples by looking up stored labels, and only switch that lookup on when labels repeat.
An early signal opens this lookup only when some label repeats. With 1 example per class the model was right about 20% of the time, and with 6 it was right 96%.
Give a model a dozen sentences tagged with made-up labels like foo and bar, and it learns the mapping without weight updates. Earlier work found parts of this, but not how they fit together.
They traced the full path in Qwen3-4B, SmolLM3-3B and Llama-3.2-3B. Each label token stores the class of its sentence, and about 90 later heads match the new sentence against those stored classes.
For few-shot prompts, repeat every label several times and pick classes the model can already tell apart, since examples cannot teach a feature it lacks.
– arxiv. org/abs/2611.05344
Title: "A Circuit for In-Context Learning"
来源：Rohan Paul · x.com
来源：RadarAI · aihot.news