用克里普克式诘难重思评估：LLM 的“意义即使用”面临规则遵循挑战

Rethinking Evaluation for Principled Generalization Using a Kripkean Challenge to Meaning as Use

AI 导读

研究借用克里普克对维特根斯坦的解读指出，大语言模型“意义即使用”的直觉观面临规则遵循挑战：有限的使用历史无法确定未来扩展的唯一规则。作者认为，测量固定用例表现或局部鲁棒性的 benchmark 本身不足以证明语义能力，并提出以抽象与递归两项能力为核心、配套实验范式来评估模型在有原则的领域扩展下能否保持规则与概念。

正文

View PDF HTML (experimental)

Abstract:Large language models operationalize an intuitive, view of meaning as use. We argue that it faces a rule-following challenge. Using Kripke's interpretation of Wittgenstein, we argue that finite histories of use do not determine a unique rule for future extension. This reframes the debate about OOD generalization but also about NLP evaluation; benchmarks that measure performance on fixed use cases or local robustness, we argue, do not by themselves establish semantic competence. To evaluate whether models preserve rules and concepts under principled domain extension, we propose to evaluate two capacities, central to this ability: abstraction and recursion; and we offer an experimental paradigm for testing both.

Subjects:	Computation and Language (cs.CL)
Cite as:	arXiv:2610.35111 [cs.CL]
	(or arXiv:2610.35111v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.35111 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Omar Naim [view email]
[v1] Mon, 26 Oct 2026 09:52:01 UTC (6,444 KB)

来源：arXiv cs.CL · arxiv.org