跳到正文
arXiv cs.AI· Chaipat Tirapongprasert, Rajarshi Das, Parijat Dube·· 10 小时前AI 评分30

研究 LLM 推理轨迹中的表征:面向迭代博弈的机制可解释性分析

Tracking Representations in LLM Reasoning Traces for Iterated Games

AI 导读

研究用机制可解释性技术分析 LLM 智能体在不完全信息迭代博弈中的决策过程,发现模型会在 CoT 中形成并反复修正对博弈的内部表征,一旦过早固守错误信念便很难恢复,这一失败同时体现在推理轨迹文本和层激活中。该工作为多智能体系统战略推理错误的诊断提供了新路径,已被 SoCalNLP 2026 接收。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large language model (LLM) agents achieve state-of-the-art performance across domains through chain-of-thought (CoT) reasoning. Despite their ubiquitous use, little is understood about the relationship between the externalized trace and the model's actual computation. We address this by investigating how LLM agents make decisions in iterative games with incomplete information. Using a series of mechanistic interpretability techniques, we find that the model forms and iteratively revises an internal representation of the game throughout the CoT. Once the model prematurely commits to an incorrect belief, it rarely recovers. This failure is manifested in both the literal trace and the layer activations. Ultimately, as multi-agent systems are more prevalent, our work opens the door to a more focused diagnosis of their strategic reasoning errors. For illustrative purposes, we offer an interactive visualizer of our methodology on selected sample traces at this https URL
Comments: Accepted to SoCalNLP 2026. 15 pages, 11 figures
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT)
Cite as: arXiv:2611.23403 [cs.CL]
  (or arXiv:2611.23403v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.23403

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Chaipat Tirapongprasert [view email]
[v1] Tue, 17 Nov 2026 19:03:18 UTC (676 KB)

来源:arXiv cs.AI · arxiv.org