arXiv cs.AI· Tianxiang Chen, Jingyuan Zhou, Guoyao Yang, Kaidi Yang·· 10 小时前AI 评分29
大语言模型上下文学习与上下文外幻觉的机制分析
A Mechanistic Analysis of In-Context Learning and Out-of-Context Hallucination
AI 导读
研究从动态上下文中的功能角色视角统一分析大语言模型的上下文学习(ICL)与上下文外(OOC)幻觉,指出角色信息可在模型内部被表征却未必可靠约束输出。作者提出一阶分解刻画角色效应,发现其显著影响依赖足够贡献幅度与符号一致聚合两个必要条件。因果分析显示角色跨度贡献影响响应结构、内容选择与事实取值等关键 token 选择,角色一致的选择依赖这些决策处足够的角色跨度支持。
正文 · 原文
Abstract:Large language models can infer and follow specific rules through in-context learning (ICL), yet may fail to remain faithful to provided evidence, giving rise to out-of-context (OOC) hallucination. Existing work has largely investigated the underlying mechanisms of ICL and OOC hallucination along separate lines. However, this separation obscures the mechanistic basis for these seemingly opposing outcomes. In this work, we study this contrast through the unified view of functional roles in dynamic context. The key observation is that role information can be internally represented without reliably constraining output. We derive a first-order decomposition to characterize the role effect, showing that its substantial influence depends on two necessary conditions, namely sufficient contribution magnitude and coherent signed aggregation. Causal analysis further shows that role span contributions affect key token choices involving response structure, content selection, and factual values. We further show that role-consistent choices depend on sufficient role span support at these key decisions, while failing choices can be reversed when this support overcomes the deficit. These findings support a common mechanistic view of ICL and OOC hallucination: each functional role defines the pattern of output decisions over which role span contributions must remain effective, and differences in this organization account for the seemingly opposing outcomes.
| Comments: | 25 pages, 6 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.24000 [cs.CL] |
| (or arXiv:2611.24000v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24000 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tianxiang Chen [view email]
[v1]
Wed, 18 Nov 2026 07:48:46 UTC (2,266 KB)
来源:arXiv cs.AI · arxiv.org