arXiv cs.AI· Xiaoou Liu, Tiejin Chen, Hanyang Chen, Yuan Ling, Hua Wei·· 12 小时前AI 评分22
EquivAug：用符号等价增强提升 RLVR 的推理路径多样性
Diverse Reasoning Paths Matter: Symbolic Equivalence Augmentation for Enhancing RL Exploration
AI 导读
研究者提出 EquivAug 数据增强框架，通过符号等价变换把 CoT 解答转成形式化符号表示再重构为自然语言，生成推理步骤不同但可证明正确的训练数据。在 GSM8K 和 FOLIO 上跨模型规模实验显示，EquivAug 在 SFT 后取得最高推理路径多样性，并在 GRPO 后取得最强准确率，表明 SFT 阶段的 CoT 多样性与下游 RLVR 性能正相关。
正文 · 原文
Abstract:Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for improving the reasoning capabilities of LLMs. However, the role of supervised fine-tuning (SFT) data in enabling effective downstream RL remains underexplored. In this paper, we investigate a key hypothesis: the diversity of reasoning steps in chain-of-thought (CoT) during SFT is a critical enabler of RLVR performance. To test this hypothesis, we introduce EquivAug, a data augmentation framework that uses symbolic equivalence transformations to generate diverse yet provably correct reasoning paths. By converting CoT solutions into formal symbolic representations, applying equivalence transformations, and reconstructing natural-language explanations, EquivAug produces training data that differs in reasoning steps rather than surface-level lexical variation. Experiments on GSM8K and FOLIO across model scales demonstrate that EquivAug consistently achieves the highest reasoning path diversity after SFT and the strongest accuracy after GRPO. Our results establish that CoT diversity during SFT is positively correlated with downstream RLVR performance.
| Comments: | Accepted by 2026 AACL |
| Subjects: | Symbolic Computation (cs.SC); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.22665 [cs.SC] |
| (or arXiv:2610.22665v1 [cs.SC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22665 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiaoou Liu [view email]
[v1]
Thu, 15 Oct 2026 21:31:42 UTC (424 KB)
来源：arXiv cs.AI · arxiv.org