arXiv cs.CL· Tom-Felix Thormann·· 11 小时前AI 评分40
研究揭示“测谎仪”方法检测 LLM 欺骗行为的盲区
Probing the Limits of the Lie Detector Approach to LLM Deception
AI 导读
一项 arXiv 研究挑战了“欺骗即说谎”的假设，构建了基于语言理论的人工编写“非虚假欺骗性陈述”数据集。在六个不同规模和家族的开源 LLM 上测试发现，部分模型能在强制选择任务中选出这类陈述，而基于标准真假数据集训练的真值探针虽能有效检测虚假输出，检测此类非虚假欺骗陈述的表现却显著更差。研究还探索了在对话语境中训练欺骗探针或加入误导性非虚假陈述能否提升检测率。
正文 · 原文
Abstract:A strand of mechanistic approaches to deception in large language models (LLMs) relies on "lie detectors" - truth probes aimed to identify model outputs that are internally represented as false. This lie detector approach to LLM deception implicitly assumes that deception is coextensive with lying. This paper challenges that assumption by experimentally investigating whether LLMs can produce deceptive statements that are not false and whether truth probes (if naively applied as deception detectors) fail to detect such behavior. In doing so, we make four core contributions: First, we provide a dataset of manually written examples of deceptive non-falsities grounded in linguistic theory. Second, we show across six open-source LLMs of different sizes and families that some models can select these deceptive non-falsities in a forced-choice task. Third, we demonstrate that while truth probes trained on a standard true-false dataset perform well at detecting false outputs, they perform significantly worse at detecting deceptive non-falsities. We view this as substantiating the worry that naive applications of truth probes as deception detectors suffer from a critical blind spot. Fourth, we explore whether training deception probes in dialogical contexts or including misleading non-false statements in the training improves the deception detection rate.
| Comments: | The dataset has been reworked and validated, experiments have been reconducted, and a broader selection of models than in the first version has been investigated |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2603.10003 [cs.CL] |
| (or arXiv:2603.10003v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2603.10003 arXiv-issued DOI via DataCite |
Submission history
From: Tom-Felix Thormann [view email]
[v1]
Mon, 16 Feb 2026 19:01:55 UTC (433 KB)
[v2]
Mon, 2 Nov 2026 16:39:23 UTC (1,046 KB)
来源：arXiv cs.CL · arxiv.org