arXiv cs.CL· Shin'ya Yamaguchi·· 13 小时前AI 评分29
用上下文感知解码缓解思维链中的查询捷径
Mitigating Query Shortcuts in Chain-of-Thought with Context-Aware Decoding
AI 导读
研究将对比解码方法 CAD 应用于 CoT 推理，提出 Rationale Contrastive Decoding（RCD），通过对比有无推理依据条件下的下一 token 分布，放大因依据而提升概率的 token。在五个推理基准上，RCD 相比标准 CoT 提升了答案准确率，干预分析显示最终答案对推理依据内容的依赖增强。该方法无需训练，为缓解查询捷径提供了简单途径。
正文 · 原文
Abstract:While Chain-of-Thought (CoT) prompting improves the reasoning performance of large language models (LLMs), recent studies have shown that final answers do not always depend on the generated rationales. In particular, LLMs may rely on query-specific shortcuts rather than the information provided by their reasoning steps. In this paper, we investigate whether context-aware decoding (CAD), an existing contrastive decoding method, can mitigate such shortcut behavior when applied to CoT reasoning. We study this application as Rationale Contrastive Decoding (RCD), which contrasts the next-token distributions conditioned on the query with and without its rationale, amplifying tokens whose likelihood increases due to the rationale. We provide a theoretical interpretation of this decoding rule via a KL-regularized reward-maximization objective. Experiments across five reasoning benchmarks demonstrate that RCD improves answer accuracy over standard CoT. Furthermore, intervention analyses using ground-truth and random rationales reveal that RCD increases the behavioral dependence of final answers on the rationale content. Our findings suggest that contrastive decoding offers a simple, training-free approach to mitigating query shortcuts.
| Comments: | 9 pages, 3 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.29518 [cs.CL] |
| (or arXiv:2610.29518v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29518 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shin'ya Yamaguchi [view email]
[v1]
Wed, 21 Oct 2026 10:05:25 UTC (245 KB)
来源：arXiv cs.CL · arxiv.org