arXiv cs.CL· Yulu An, Yifei Feng, Xiangyang Yin·· 12 小时前AI 评分32
SpecCompact：用槽位并行语义推测验证加速 LLM 智能体上下文压缩
SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification
AI 导读
SpecCompact 通过微调小型 drafter 并行生成摘要槽位、由目标模型逐行验证并修复被拒内容，在 AppWorld 和 τ³ 两个基准、两种目标模型上相较 Hermes 式摘要实现 2.04–5.63× 压缩加速。Hermes 内的用户研究显示中位压缩加速达 8.68×，但下游评测显示性能权衡因模型和任务而异。
正文 · 原文
Abstract:LLM agents use context compaction to manage growing interaction histories, but summarizing long histories with large models can introduce substantial delays. Compaction therefore requires balancing information retention against latency. We observe that structured summary slots can retain information when generated independently. Based on this observation, we present SpecCompact, a framework that accelerates compaction through slot-parallel semantic speculation. SpecCompact fine-tunes a small drafter and generates slots in parallel. The target model verifies each proposed line and repairs rejected content, while subsequent drafting uses the updated slot prefix. System optimizations further reduce repeated context processing and model calls. Across AppWorld and $\tau^3$ benchmarks with two target models, SpecCompact achieves $2.04$--$5.63\times$ compaction speedups over Hermes-style summarization. Separate downstream evaluations reveal model- and task-dependent performance trade-offs. A user study within Hermes further shows an $8.68\times$ median compaction speedup.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.06876 [cs.CL] |
| (or arXiv:2611.06876v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06876 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiangyang Yin [view email]
[v1]
Wed, 4 Nov 2026 18:34:41 UTC (203 KB)
来源：arXiv cs.CL · arxiv.org