arXiv cs.CL· Yujing Wang, Yulan He, Wenya Wang, Hanqi Yan·· 12 小时前AI 评分31
VERITAS：用逐层 Jacobian 投影将模型计算语言化，检测 CoT 推理忠实性
Verbalising Model's Computation for Chain-of-Thought Faithfulness Detection
AI 导读
VERITAS 通过逐层 Jacobian 投影，在每个生成的 CoT token 上把模型内部推理演化转写为可读文本，并用提取特征训练分类器定位推理偏离忠实性的位置。在 FaithCoT-Bench 与 BonaFide 的四个数据集上，其平衡准确率较 LLM-as-Judge 提升 28.4%，较激活探测平均提升 16.0%，并可泛化到不同数据集与模型架构。
正文 · 原文
Abstract:Current Chain-of-Thought (CoT) verification methods detect reasoning faithfulness from model outputs or static activations, but are limited in explaining how unfaithfulness develops during generation. We introduce VERITAS to verbalise the model's computation: how different reasoning anchors emerge, transfer across layers, and surface in the output. VERITAS adopts the layer-wise Jacobian projection to produce a verbalised account of the model's internal reasoning evolution at each generated CoT token. By training a classifier on the extracted features along the traces, we show that these structured features provide insightful perspectives on how and where reasoning diverges toward unfaithfulness. Experiments on four datasets from FaithCoT-Bench and BonaFide show that VERITAS achieves a significant performance gain over LLM-as-Judge, with a 28.4% gain in balanced accuracy, and an average improvement of 16.0% over activation probing. Further analysis shows VERITAS can be generalised to different datasets and model architectures. Our work shows that verbalising model computation provides a richer and more interpretable lens on CoT faithfulness, moving beyond detection toward a mechanistic understanding of how reasoning fails.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.23275 [cs.CL] |
| (or arXiv:2610.23275v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23275 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yujing Wang [view email]
[v1]
Fri, 16 Oct 2026 10:44:23 UTC (435 KB)
来源：arXiv cs.CL · arxiv.org