arXiv cs.AI· Samkit Jain, Sai Kishan Pampana, Shlok Agarwal·· 12 小时前AI 评分34
MLIPS：多层索引器池共享方法
MLIPS: Multi-Layer Indexer Pool Sharing
AI 导读
MLIPS 是一种免训练的多层索引器池共享方法，由生产者层发布大于注意力预算 K 的 top-M 候选池，后续层从中各自选取 top-K，从而共享候选搜索但保留独立打分与选择。
正文 · 原文
Abstract:Sparse attention limits the keys used by each query, but finding those keys can still require a full-prefix scan at every layer. We introduce \textbf{MLIPS (Multi-Layer Indexer Pool Sharing)}, a training-free method that shares candidate search while retaining each consumer's own scoring and selection. A producer publishes a top-$M$ pool larger than the attention budget $K$; subsequent layers select their own top-$K$ from that pool. Measurements across three sparse-attention model families show substantially higher coverage from broad pools than overlap between final selections. We implement layerwise sharing, persistent decode reuse, and a coarse producer in vLLM for DeepSeek-V4-Flash. Across four long-context suites, three consumer layers per producer give $1.14$--$1.40\times$ speedups and $14.8$--$27.4\%$ lower GPU energy, with observed accuracy changes of $-1.49$ to $+0.29$ percentage points in single runs. More extensive sharing exposes task-dependent quality losses. Sampled prefill traces also reveal recurring selected KV positions across query groups and layers. We characterize this shared working-set structure and use it to motivate Top-$G$, an exploratory bounded pool construction for prefill. Its deployment, task accuracy, and serving benefits remain open.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Performance (cs.PF) |
| Cite as: | arXiv:2610.27438 [cs.LG] |
| (or arXiv:2610.27438v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.27438 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shlok Agarwal [view email]
[v1]
Tue, 20 Oct 2026 00:37:29 UTC (1,593 KB)
来源：arXiv cs.AI · arxiv.org