arXiv cs.CL· Tianjun Zhong, Linyang He, Ziyang Li, Nima Mesgarani·· 12 小时前AI 评分34
LLM 用 DAG 推理：探测推理的潜在图结构
LLMs Think with DAGs: Probing the Latent Graph Structure of Reasoning
AI 导读
研究提出 Reasoning DAG Probing，将每个前提、中间结论和最终答案形式化为 DAG 节点，用轻量探针从 LLM 隐藏状态预测节点深度、成对距离和邻接关系。在逻辑、数学和代码推理基准上，DAG 结构在中间层编码最强，后层更倾向恢复深层节点和长程依赖，且随模型规模增强。将激活导向真实 DAG 结构可将部分原本错误的生成转为正确。
正文 · 原文
Abstract:Large language models often externalize reasoning as linear chains, but many reasoning problems can be more naturally modeled as directed acyclic graphs (DAGs), where intermediate conclusions branch, merge, and are reused. Whether such graph structure is reflected in model internals remains unclear. We introduce Reasoning DAG Probing to answer this question. We formalize each premise, intermediate conclusion, and final answer as a DAG node, and train lightweight probes to predict node depth, pairwise distance, and adjacency from LLM hidden states. Across logical, mathematical, and code reasoning benchmarks, we find that DAG structure is meaningfully encoded in LLM representations: it peaks in intermediate layers, with later layers preferentially recover deeper nodes and longer-range reasoning dependencies, and becomes stronger with model scale. DAG structure also progressively emerges during autoregressive generation in ways that anticipate answer correctness, while steering activations toward the ground-truth DAG structure can convert some originally incorrect generations into correct ones. These results suggest that LLM reasoning is not purely sequential, but reflects measurable latent graph structure.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2601.17593 [cs.CL] |
| (or arXiv:2601.17593v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2601.17593 arXiv-issued DOI via DataCite |
Submission history
From: Tianjun Zhong [view email]
[v1]
Sat, 24 Jan 2026 21:11:51 UTC (650 KB)
[v2]
Sun, 5 Apr 2026 23:42:54 UTC (1,903 KB)
[v3]
Fri, 7 Aug 2026 20:09:31 UTC (918 KB)
[v4]
Thu, 15 Oct 2026 18:09:02 UTC (653 KB)
来源：arXiv cs.CL · arxiv.org