arXiv cs.AI· Peter Chen, Ni Zhang, Ying Xu, Siyuan Zhang·· 1 天前AI 评分44
基于LLM的智能体系统25个认知架构模块审计：18个模块的计算值从未到达生产输出
Real Computation or Merely Naming? A Pre-Registered Mechanism Audit of 25 Cognitive-Architecture Modules in an LLM-Based Agent System
AI 导读
一项预注册机制审计发现，某LLM智能体系统宣称的25个认知架构模块中，18个模块的计算值从未到达任何生产输出，并识别出三种新的空转接线形态。研究对11个模块执行干预，61条冻结标准中53条确认、5条被证伪；修复实验将规划器任务成功率从0.028提升至1.000，状态控制器误差降低52%。
正文 · 原文
Abstract:Agent systems built on local large language models are increasingly marketed through the vocabulary of cognitive architecture: modules named global workspace, theory of mind, causal reasoning, or active inference are listed as features, and the presence of the name is taken as the presence of the mechanism. We ask the prior question -- does the named computation actually happen -- and contribute a transferable verification method for the claim-implementation gap: a four-question static audit protocol with line-number evidence and call-chain tracing, demonstrated on two systems. We audit a system claiming 25 cognitive mechanisms, pre-register falsifiable predictions for 11 modules and execute the interventions, replicate the protocol on the IIT-phi core of a second, independent system with its own pre-committed falsification criteria, and run two repair experiments as positive controls. In 18 of 25 modules the computed value never reaches any production output; three idle-wiring shapes new to the dead-code and code-smell literatures emerge (connected-but-never-called, called-but-inert, called-but-content-blind). Of 61 frozen criteria across 12 interventions, 53 were confirmed, two partially adjudicated, five falsified, one adjudication-ambiguous. Repairs raised planner task success from 0.028 to 1.000 and cut state-controller error by 52%. Module names and module computations must be audited as independent claims; the protocol and the five-class failure taxonomy are reusable for the claim-implementation gap at two-system, case-level depth.
| Comments: | 40 pages. Static audit of 25 cognitive-architecture modules (18 carry no computed value to any production output), pre-registered interventions on 11 modules, 61 frozen criteria (53 confirmed / 5 falsified), two repair controls (0.028 -> 1.000 task success; -52% error). Artifact: this https URL |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.41604 [cs.AI] |
| (or arXiv:2610.41604v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41604 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhang Siyuan [view email]
[v1]
Fri, 30 Oct 2026 11:38:04 UTC (25 KB)
来源：arXiv cs.AI · arxiv.org