arXiv cs.CL· Kirill Afendulev, Alexey Dontsov, Alla Chepurova, Elena Tutubalina, Anton Korznikov·· 12 小时前AI 评分31
上下文还是知识：LLM 如何归因自己的答案
Context or Knowledge: How LLMs Attribute Their Own Answers
AI 导读
研究通过干预生成答案后、提问来源前的 KV cache，考察 LLM 如何报告答案来源。在 7B-32B 四个模型上，改动或删除文档 cache 从不改变来源报告，而移植问答（Q+A）cache 会让报告完全跟随供体。沿拟合方向操控 Q+A 值可逆转 98.9-100% 的报告，随机方向最多只翻转 1.1%。
正文 · 原文
Abstract:Language models can answer a question from a supplied document or from their parametric knowledge. When later asked where the answer came from, what determines their source report? We study this by intervening on the model's key-value (KV) cache after the answer is generated and before the source question is asked. On a controlled factual task with four models (7B-32B), changing or removing the document's cache never changes the report. In contrast, transplanting the question-and-answer (Q+A) cache between runs with identical question and answer tokens transfers it: every eligible report follows the donor, while the recipient's document is unchanged. On the two smaller models, steering Q+A values along directions fitted before any source question reverses 98.9-100% of reports across four document conditions at a fixed strength, while norm-matched random directions flip at most 1.1%. The components of keys and values along fitted directions carry the transplant effect: copying them alone nearly reproduces the report change; holding them fixed nearly prevents it. Applied during answer generation, the same value directions also change which source supplies the answer, though more weakly and depending on the prompt. Source reporting in this setting thus depends causally on cached state produced while answering. We do not establish general reporting accuracy or a complete interpretation of this state.
| Comments: | 25 pages, 7 figures, 23 tables, including appendices |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.32109 [cs.CL] |
| (or arXiv:2610.32109v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32109 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kirill Afendulev [view email]
[v1]
Fri, 23 Oct 2026 03:21:36 UTC (38 KB)
来源：arXiv cs.CL · arxiv.org