跳到正文
arXiv cs.AI· Zijun Gao, Zhikun Xu, Xiao Ye, Ben Zhou·· 4 小时前AI 评分38

CORE:面向概念强化的数学推理框架,弥合定义与应用鸿沟

CORE: Concept-Oriented Reinforcement for Bridging the Definition-Application Gap in Mathematical Reasoning

AI 导读

研究者提出 CORE(Concept-Oriented REinforcement)强化学习训练框架,将显式概念转化为可控监督信号,以弥合 LLM 在数学推理中"能复述定义却不会应用"的鸿沟。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large language models (LLMs) often solve challenging math exercises yet fail to apply the concept right when the problem requires genuine understanding. Popular Reinforcement Learning with Verifiable Rewards (RLVR) pipelines reinforce final answers but provide little fine-grained conceptual signal, so models improve at pattern reuse rather than conceptual applications. We introduce CORE (Concept-Oriented REinforcement), an RL training framework that turns explicit concepts into a controllable supervision signal. Starting from a high-quality, low-contamination textbook resource that links verifiable exercises to concise concept descriptions, we run a sanity probe showing LLMs can restate definitions but fail concept-linked quizzes, quantifying the conceptual reasoning gap. CORE then (i) synthesizes concept-aligned quizzes, (ii) injects brief concept snippets during rollouts to elicit concept-primed trajectories, and (iii) reinforces conceptual reasoning via trajectory replacement after group failures, a lightweight forward-KL constraint that aligns unguided with concept-primed policies, or standard GRPO directly on concept-aligned quizzes. Across several models, CORE delivers consistent gains over vanilla and SFT baselines on both in-domain concept-exercise suites and diverse out-of-domain math benchmarks. CORE unifies direct training on concept-aligned quizzes and concept-injected rollouts under outcome regularization. It provides fine-grained conceptual supervision that bridges problem-solving competence and genuine conceptual reasoning, while remaining algorithm- and verifier-agnostic.
Comments: Accepted at ICLR 2026. Code: this https URL
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2512.18857 [cs.AI]
  (or arXiv:2512.18857v4 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2512.18857

arXiv-issued DOI via DataCite

Submission history

From: Zijun Gao [view email]
[v1] Sun, 21 Dec 2025 19:01:35 UTC (157 KB)
[v2] Mon, 2 Mar 2026 21:29:46 UTC (924 KB)
[v3] Thu, 7 May 2026 11:50:40 UTC (1,137 KB)
[v4] Tue, 17 Nov 2026 06:48:39 UTC (297 KB)

来源:arXiv cs.AI · arxiv.org