arXiv cs.CL· Nathani\"el de Leeuw, Marceau Nahon, Mathis Reymond, Raja Chatila, Mehdi Khamassi·· 13 小时前AI 评分42
语义欺骗：推理模型为何算不出一道加法题
Semantic Deception: When Reasoning Models Can't Compute an Addition
AI 导读
研究提出“语义欺骗”实验框架，用全新符号重新定义数字和数学运算符，测试LLM能否摆脱误导性语义联想、维持符号抽象。对四个LLM的实验显示，语义线索会显著拉低推理模型在极简任务上的表现，链式推理可能放大模型对统计相关性的依赖。
正文 · 原文
Abstract:Large language models (LLMs) are increasingly used in situations where human values are at stake, such as decision-making tasks that involve reasoning when performed by humans. We investigate the so-called reasoning capabilities of LLMs over novel symbolic representations by introducing an experimental framework that tests their ability to process and manipulate unfamiliar symbols. We introduce semantic deceptions: situations in which symbols carry misleading semantic associations due to their form, such as being embedded in specific contexts, designed to probe whether LLMs can maintain symbolic abstraction or whether they default to exploiting learned semantic associations. We redefine standard digits and mathematical operators using novel symbols, and task LLMs with solving simple calculations expressed in this altered notation. The objective is: (1) to assess LLMs' capacity for abstraction and manipulation of arbitrary symbol systems; (2) to evaluate their ability to resist misleading semantic cues that conflict with the task's symbolic logic. Through experiments with four LLMs we show that semantic cues can significantly deteriorate reasoning models' performance on very simple tasks. They reveal limitations in current LLMs' ability for symbolic manipulations and highlight a tendency to over-rely on surface-level semantics, suggesting that chain-of-thoughts may amplify reliance on statistical correlations. Even in situations where LLMs seem to correctly follow instructions, semantic cues still impact basic capabilities. These limitations raise ethical and societal concerns, undermining the widespread and pernicious tendency to attribute reasoning abilities to LLMs and suggesting how LLMs might fail, in particular in decision-making contexts where robust symbolic reasoning is essential and should not be compromised by residual semantic associations inherited from the model's training.
| Comments: | 22 pages, 5 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2512.20812 [cs.CL] |
| (or arXiv:2512.20812v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2512.20812 arXiv-issued DOI via DataCite |
Submission history
From: Marceau Nahon [view email]
[v1]
Tue, 23 Dec 2025 22:22:18 UTC (407 KB)
[v2]
Wed, 21 Oct 2026 18:53:21 UTC (318 KB)
来源：arXiv cs.CL · arxiv.org