arXiv cs.CL· Zizhuo Lin, Quanling Liu, Jinsheng Quan, Chao Zhang, Yifan Zhu, Xing Shi, Jingtao Xu, Zhihui Li, Yawei Luo·· 12 小时前AI 评分31
相同证据不同答案：面向多轮语言模型的规范上下文同策略蒸馏（CCOPD）
Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models
AI 导读
研究者提出 Canonical-Context On-Policy Distillation（CCOPD），用同一基座模型分别充当基于完整提示词的冻结教师和逐轮接收证据的可训练学生，对齐学生自身轨迹与教师全上下文行为。
正文 · 原文
Abstract:Large language models (LLMs) often solve a task when all instructions are given in a single prompt, but fail when the same information is revealed gradually across turns. When a clean FULL prompt and a RAW-SHARDED conversation contain the same complete user evidence, the model should still arrive at the same answer. We argue that a key reason for this gap is self-anchored drift: responses produced under partial information introduce unsupported assumptions, and those assumptions later distort the final answer. To reduce this effect, we propose Canonical-Context On-Policy Distillation (CCOPD). During training, the same base model is used in two roles: a frozen teacher conditioned on the clean FULL prompt and a trainable student that receives the same evidence incrementally through a multi-turn conversation; CCOPD aligns the student's behavior on its own trajectories with the teacher's canonical full-context behavior. Trained only on math problem conversations, CCOPD yields a 32\% average relative improvement in RAW-SHARDED performance over the original base model across math and five zero-shot out-of-domain task families, while largely preserving full-context performance. Further analyses suggest that CCOPD strengthens grounding in user evidence and reduces sensitivity to contamination from earlier assistant turns.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.30251 [cs.CL] |
| (or arXiv:2605.30251v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.30251 arXiv-issued DOI via DataCite |
Submission history
From: Zizhuo Lin [view email]
[v1]
Thu, 28 May 2026 17:14:29 UTC (860 KB)
[v2]
Tue, 20 Oct 2026 08:59:15 UTC (1,512 KB)
来源：arXiv cs.CL · arxiv.org