arXiv cs.CL· Mehrdad Saberi, Keivan Rezaei, Soheil Feizi·· 2 天前AI 评分38
SpecHop:用连续推测加速多跳检索智能体
SpecHop: Continuous Speculation for Accelerating Multi-Hop Retrieval Agents
AI 导读
SpecHop 是一个连续推测框架,通过维护多条推测线程、异步验证预测的工具观测结果并回滚错误分支,在不改变模型最终轨迹的前提下降低多跳工具调用的实际延迟。在检索增强的多跳任务上,SpecHop 与理论预测高度吻合,部分设置下延迟最多降低 40%。该工作已被 NeurIPS 2026 接收。
正文 · 原文
Abstract:Large language models increasingly use external tools such as web search and document retrieval to solve information-intensive tasks. However, multi-hop tool use in complex tasks introduces substantial latency, since the model must repeatedly wait for tool observations before continuing. We study how to accelerate such trajectories without changing the final trajectory the model would have taken without acceleration, assuming access to faster but less reliable speculator tools. We develop a theoretical framework for lossless speculation in multi-hop tool-use settings, characterizing the optimal achievable latency gain. We propose SpecHop, a continuous speculation framework that maintains multiple speculative threads, verifies predicted observations asynchronously as target tool outputs arrive, commits correct branches, and rolls back incorrect ones. This preserves accuracy while reducing wall-clock latency. We show that SpecHop can approach oracle latency gains with enough active threads. Empirically, on retrieval-augmented multi-hop tasks, SpecHop closely matches theoretical predictions and reduces latency by up to 40\% in some settings. Code: this https URL
| Comments: | Accepted at the 40th Conference on Neural Information Processing Systems (NeurIPS 2026) |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.21965 [cs.CL] |
| (or arXiv:2605.21965v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.21965 arXiv-issued DOI via DataCite |
Submission history
From: Mehrdad Saberi [view email]
[v1]
Thu, 21 May 2026 03:55:47 UTC (488 KB)
[v2]
Fri, 13 Nov 2026 16:50:55 UTC (491 KB)
来源:arXiv cs.CL · arxiv.org