轨迹触发的稀疏协同解码：用轨迹形状特征决定何时调用目标模型

View PDF HTML (experimental)

Abstract:Dense assistance methods such as speculative decoding repeatedly verify draft tokens with a target model, keeping target computation coupled to generation. We study sparse collaborative decoding, where a draft model generates most tokens and a target model is invoked only for short corrective segments. The key problem is deciding when to call the target. We show that local uncertainty is a weak trigger: entropy, negative log-likelihood, logit margin, top-$k$ mass, and velocity features give near-chance failure prediction, with AUC 0.50-0.53. In contrast, aggregate trajectory-shape features over the draft confidence path reach AUC 0.86 using a lightweight logistic predictor trained only from draft correctness labels. This induces a trigger-timing tradeoff: early calls have more leverage but weaker evidence, while later calls have stronger evidence but less ability to redirect the trajectory. Used online, learned sparse collaborative decoding improves over the best open-loop schedule by +5.3 to +20.0 accuracy points across five GSM8K draft-target pairs. On MATH-500, it gains 7.0-13.5 points when early prediction and target competence align, but loses when either is weak. On StrategyQA, it improves over open-loop by 5.0-8.3 points, showing transfer beyond arithmetics.

Comments:	17 pages. Accepted at the EMNLP 2026 uncertainty workshop
Subjects:	Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as:	arXiv:2610.23613 [cs.CL]
	(or arXiv:2610.23613v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.23613 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Amit Trivedi [view email]
[v1] Fri, 16 Oct 2026 15:24:05 UTC (1,949 KB)