arXiv cs.CL· Dicong Yu, Wenhua Wang, Tian Wang·· 11 小时前AI 评分31
Stepwise Querying：无需访问教师 CoT 的推理蒸馏方法
Stepwise-Querying: Distilling Reasoning without Access to Teacher Chain-of-Thought
AI 导读
Stepwise Querying 通过向黑盒教师模型分步独立提问，获取中间推导、有效性检查与修正，再组装成解答用于标准监督微调，无需访问教师内部 CoT。在两个教师、两个学生和四个数学与代码基准的 16 组对比中，该方法在 14 组取得最高分。作者还提出诊断指标 SEIG，GSM8K 分析显示错误答案轨迹若保留有效中间推理仍可有用。
正文 · 原文
Abstract:Reasoning distillation is widely used to transfer the reasoning capabilities of large language models to smaller students through teacher-generated supervision. In the black-box setting, however, teachers may expose only final answers and brief solutions, leaving intermediate calculations and validity conditions unavailable and thus limiting the reasoning supervision available for distillation. We propose Stepwise Querying, which elicits step-specific explanations without requiring access to the teacher's internal chain-of-thought (CoT) traces. Independent queries to the same teacher request intermediate derivations, validity checks, and corrections for individual steps, the assembled solutions are then used for standard supervised fine-tuning. Across two teachers, two students, and four mathematics and code benchmarks, our method achieves the highest score in 14 of 16 comparisons with three baselines. We further introduce Stepwise Explanation Information Gain (SEIG), a diagnostic based on the log-ratio of a frozen student's probabilities of correct step completion under stepwise and global explanations. Analyses on GSM8K show that incorrect-answer trajectories can remain useful when they preserve valid intermediate reasoning, while further decomposition yields limited additional gains.
| Comments: | 15 pages, 7 figures, including appendices |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.41769 [cs.CL] |
| (or arXiv:2610.41769v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41769 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Dicong Yu [view email]
[v1]
Fri, 30 Oct 2026 14:10:09 UTC (1,167 KB)
来源：arXiv cs.CL · arxiv.org