arXiv cs.AI· Siyue Chen, Yifu Guo, Yuquan Lu, Zishan Xu, Jiaye Lin, Jianbo Lin, Siyu Zhang, Cheng Yang, Junxin Li, Yujia Li, Yu Huo, Ruixuan Wang·· 3 天前AI 评分33
LLM 代码推理的内部生命周期：从"酝酿"到四种解析结果
From Brewing to Resolution: Tracing the Internal Lifecycle of Code Reasoning in LLMs
AI 导读
研究提出双诊断框架，结合逐层线性探针与 Context-Stripped Decoding（CSD），在 Qwen、Llama、DeepSeek 等 16 个模型、六类代码推理任务上追踪代码推理的内部生命周期：模型先"酝酿"答案，使其在可自解码前多个层即可线性恢复，随后分化为 Resolved、Overprocessed、Misresolved、Unresolved 四种结果。
正文 · 原文
Authors:Siyue Chen, Yifu Guo, Yuquan Lu, Zishan Xu, Jiaye Lin, Jianbo Lin, Siyu Zhang, Cheng Yang, Junxin Li, Yujia Li, Yu Huo, Ruixuan Wang
Abstract:Standard accuracy metrics cannot explain why LLMs handle variable tracking but fail on semantically equivalent loops. We study an internal lifecycle of code reasoning in which models first brew the answer, making it linearly recoverable many layers before it becomes self-decodable, and then diverge into one of four resolution outcomes: Resolved, Overprocessed, Misresolved, or Unresolved. Understanding this lifecycle matters because similar task accuracies can mask fundamentally different failure modes that surface-level evaluation cannot detect. We introduce a dual diagnostic framework pairing layer-wise linear probing with Context-Stripped Decoding (CSD) and apply it to six code-reasoning task families across 16 models spanning Qwen, Llama, and DeepSeek architectures. All four outcomes carry substantial mass in every task family: overall Resolved is only 41.5%, with multiple tasks below 30%. Controlled sweeps over structure, depth, and operators expose task-specific failure bottlenecks: Function Call Resolved plunges from 61.1% to 2.5% as call depth increases from one to three. Across architectures and scales, the brewing scaffold remains stable, with normalized brewing duration 24-42% across all 16 models, while resolution success varies with capability. This indicates that the scaffold is a stable empirical regularity across the tested decoder-only Transformer families, whereas resolution success covaries with capability, scale, and training. Code: this https URL
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2606.17648 [cs.AI] |
| (or arXiv:2606.17648v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.17648 arXiv-issued DOI via DataCite |
Submission history
From: Yifu Guo [view email]
[v1]
Tue, 16 Jun 2026 08:06:50 UTC (1,300 KB)
[v2]
Wed, 26 Aug 2026 08:06:56 UTC (1,285 KB)
[v3]
Sun, 11 Oct 2026 03:37:01 UTC (2,437 KB)
来源：arXiv cs.AI · arxiv.org