arXiv cs.CL· Shailesh Rana·· 12 小时前AI 评分38
语言模型如何在内部收敛到答案：Llama、Qwen、Gemma 四模型研究
The Shape of Wisdom: How Language Models Converge on an Answer
AI 导读
研究者在 Llama-3.1-8B、Qwen2.5-7B、Gemma-2-9B 和 Llama-3.2-1B 四个开源模型上输入 2,256 个事实性问题，发现同答案问题会在状态空间中收敛到共享的“答案质心”，且答案在网络 47%-71% 深度处的某一层突然变得可读。网络后段将答错问题的状态移到正确答案质心，可使模型在 83%-97% 的情况下改答正确，而同等幅度随机移动仅 2%-4% 有效。
正文 · 原文
Abstract:When a language model answers a factual question, where does its internal state go? We asked four open models (Llama-3.1-8B, Qwen2.5-7B, Gemma-2-9B and Llama-3.2-1B) 2,256 factual questions whose answers recur, such as the country a person is a citizen of. At every layer, we recorded the state from which the answer is produced. Questions that receive the same answer converge on a shared region of state space, which we summarise by its centre, the answer centroid. The answer becomes readable within a single layer. In every model, the share of questions whose state lies nearest its own answer's centroid jumps at one layer, between 47% and 71% of the way through the network. The centroids follow the answer the model gives rather than the question: the centroid of a country learned from questions about people's citizenship also identifies that country in questions about towns and films. The model also uses the centroids. After the jump, moving the state of a wrongly answered question to the right answer's centroid makes the model give the right answer in 83%-97% of cases late in the network, against 2%-4% for a random move of the same size. Before the jump, moves of the same size almost never work. Near misses, in which the model answers wrongly but picks the right option in multiple choice, can be seen from the onset layer, where the right answer's centroid is already among the nearest to the state. This signal is about as informative as the right answer's rank in the model's final output, and more informative than the model's confidence in its reply.
| Comments: | 10 pages, 5 figures. This version replaces the earlier decision-trajectory analysis with a new study. Code and data: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.01202 [cs.AI] |
| (or arXiv:2606.01202v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.01202 arXiv-issued DOI via DataCite |
Submission history
From: Shailesh Rana [view email]
[v1]
Sun, 31 May 2026 12:33:36 UTC (157 KB)
[v2]
Tue, 20 Oct 2026 10:43:08 UTC (345 KB)
来源：arXiv cs.CL · arxiv.org