arXiv cs.CL· Auss Abbood, Fernando Mart\'inez-Plumed, Jos\'e Hern\'andez-Orallo, Nigel Collier·· 13 小时前AI 评分34
研究质疑 LLM 时间推理的“算术-语义”二分法：知识才是主要驱动力
Keeping Time vs. Understanding Time: Neither Alone Explains Temporal Reasoning in LLMs
AI 导读
一项研究直接检验了 LLM 时间推理分为时间算术与时间语义两类的假设，发现按此二分微调后增益主要是数据集特定的（+6.8 至 +23.1 个百分点），同模态跨数据集迁移仅 -3.2 至 +1.1 个百分点。基于认知需求的模型评估显示，定量与逻辑推理并非这些基准表现的主要驱动力，知识才是。这表明当前时间推理基准并未清晰实现算术-语义区分，更多是在奖励模型记住时间通常如何运作。
正文 · 原文
Abstract:Temporal reasoning is widely assumed to decompose into two modes: temporal arithmetic (date and time calculations), and temporal semantics (understanding temporal and causal relations). This dichotomy shapes how benchmarks are designed, how models are trained, and how progress is measured, yet its empirical validity has never been directly tested. We provide two lines of evidence that challenge it. First, we test whether tasks labelled as arithmetic or semantic by the benchmarks support transferable gains within their purported mode. Finetuning on these subsets yields predominantly dataset-specific gains ($+6.8$ to $+23.1$ percentage points), while transfer to a different dataset in the same mode ranges from $-3.2$ to $+1.1$ percentage points. Across the two benchmark families used in the transfer analysis, this pattern provides little evidence of coherent, transferable temporal abilities. Second, using cognitive-demand-based model evaluation, we examine which capabilities actually drive performance on these benchmarks. We find that quantitative and logical reasoning, the assumed proxies for temporal arithmetic and temporal semantics, are not the primary drivers; knowledge is. Together, these findings show that current temporal reasoning benchmarks do not cleanly realise the arithmetic-semantics distinction and instead largely reward models for remembering how time typically works.
| Comments: | 30 pages, 9 figures, 24 tables (including appendices) |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.29507 [cs.CL] |
| (or arXiv:2610.29507v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29507 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Auss Abbood [view email]
[v1]
Wed, 21 Oct 2026 09:58:46 UTC (364 KB)
来源：arXiv cs.CL · arxiv.org