arXiv cs.CL· Adarsh Singh, Aman Singh, Kushal Raj Bhandari, Soham Dan, Anoop Kumar, Vivek Gupta·· 12 小时前AI 评分31
EKLAV：仅用答案监督、将教师思维链作为特权上下文的推理蒸馏方法
EKLAV: Elicited Key CoT Learning via Answer only superVision
AI 导读
EKLAV 提出一种推理蒸馏方法：把教师的思维链保留在输入中作为特权上下文，但只用最终答案做监督。在 BRIGHT 段落重排上，它在两个模型族的四个测试检查点均提升平均 nDCG@10，48 组模型-领域对比中胜出 41 组，训练算力最多降低 31.6%、推理时间最多降低 32.7%。在四个留出表格重排基准中三个取得最优，平均超过全轨迹蒸馏 3.0 分、超过无推理的纯答案训练 5.5 分。
正文 · 原文
Abstract:Reasoning distillation trains a student to reproduce both the teacher's chain-of-thought trace and its final answer. We ask whether the trace is better used as privileged training context than as an imitation target. EKLAV keeps the teacher's reasoning in the input sequence but supervises only the final task answer, applying a task-dependent placement that moves part or all of the trace into the prompt. On BRIGHT passage reranking, EKLAV improves average nDCG@10 at all four tested checkpoints across two model families, winning 41 of 48 model-domain comparisons while reducing training compute by up to 31.6% and inference time by up to 32.7%. On four held-out table-reranking benchmarks, it is the best of four conditions on three, exceeding full-trace distillation by 3.0 points and answer-only training without reasoning by 5.5 points on average. Analysis of generated traces reveals that EKLAV students defer their first verbalized verdict later (0.76 vs. 0.44 relative position), reverse fewer decisions (5% vs. 16%), and produce reasoning more predictive of their own rankings than STD-COT's, though less predictive than the base model's. The recipe reuses a fixed teacher corpus, requires no student rollouts or iterative teacher feedback, and applies to both pointwise and listwise formulations. Code and Data are at this https URL
| Comments: | 8 pages, 3 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2607.25182 [cs.CL] |
| (or arXiv:2607.25182v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.25182 arXiv-issued DOI via DataCite |
Submission history
From: Adarsh Singh [view email]
[v1]
Tue, 28 Jul 2026 01:23:04 UTC (881 KB)
[v2]
Fri, 16 Oct 2026 09:04:49 UTC (2,244 KB)
来源：arXiv cs.CL · arxiv.org