跳到正文
arXiv cs.AI· Ilia Karpov, Andrey Atamanyuk, Dmitry Ilvovsky, Aleksandr Anokhin, Alex Elizov, Dmitry Zagorulkin, Sergei O. Kuznetsov·· 10 小时前AI 评分31

用结构化表示建模 LLM 推理动态

Modeling LLM Reasoning Dynamics with Structural Representations

AI 导读

研究提出 T/E/S 结构化表示,将语言模型推理拆解为文本、局部事件与活动跨度,在推理与信息检索任务中,事件与跨度使 Transformer 的下一结果预测在 76 个评估组中 70 组得到改善,Qwen 配合 LoRA 则为 74 组。

正文 · 原文

View PDF HTML (experimental)

Abstract:Language-model reasoning alternates between sustained activities and local operations. We study how this temporal structure predicts continuation and the computation needed to analyze it. T/E/S combines text, local events, and spans of activity. Across reasoning and information-seeking tasks, events and spans improve next-outcome prediction in 70 of 76 evaluation groups for a Transformer and 74 of 76 for Qwen with LoRA. Span duration and preceding events are associated with transitions, and retaining event history across span boundaries improves prediction. We formulate budget-allocation counting and action-conditional continuation queries, prove counting hardness under explicit encodings, and derive conditions for affordable decision certification. In finite-process experiments, bounds that cancel shared continuation mass recover the same optimal actions as full dynamic programming while reducing mean inference CPU time by 77.7%. On 71 held-out HotpotQA and MuSiQue questions, a compact structural observer improves prediction of whether a fixed inference procedure finishes within a CPU allowance: Brier loss falls from 0.0836 to 0.0756. An annotation audit identifies substantial errors in that observer, separating this computational result from semantic validation of its features.
Comments: 8 pages, ACM Web Conference (WWW '27) format
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.23749 [cs.AI]
  (or arXiv:2611.23749v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.23749

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Ilia Karpov [view email]
[v1] Wed, 18 Nov 2026 03:02:29 UTC (57 KB)

来源:arXiv cs.AI · arxiv.org