Attention-Mass Condensation 如何用于稀疏解码

View PDF HTML (experimental)

Abstract:Attention-mass concentration creates an opportunity for sparse decoding, but retained mass alone does not guarantee a stable greedy decision: retrieval error, omitted value directions, and recursive decoding all matter. We formalize this distinction with an exact omitted-mass identity and a sufficient downstream margin condition, then characterize a query-dependent mean-pooled block selector. On Qwen2-0.5B, a paired fresh-selection sweep covers supports of 97--769 positions, contexts of 2K--16K, and five prefixes per context. The primary exact-match result is that none of 60 runs remains identical to dense decoding through 128 tokens. Distributional quality is distinct: for supports of at least 193, seven of nine context-support conditions have median teacher-forced continuation perplexity changes within 5\% of dense, but prompt-level ranges include severe 16K outliers. All seven runs with teacher-forced match below 70\% have perplexity increases above 100\%; these observations come from two prefixes and suggest a warning regime, not a general threshold. The measured perplexity is teacher-forced on the dense model's own continuation, not the sparse model's free-running output. Separate retrieval and attention-mass probes illustrate why captured mass alone is not a retrieval or decision guarantee. Isolated operator timings do not establish matched-quality acceleration or end-to-end serving speed.

Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as:	arXiv:2602.06317 [cs.LG]
	(or arXiv:2602.06317v3 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2602.06317 arXiv-issued DOI via DataCite
Related DOI:	https://doi.org/10.5281/zenodo.18383733 DOI(s) linking to related resources

Submission history

From: Jorge Ruiz Williams [view email]
[v1] Fri, 6 Feb 2026 02:32:42 UTC (12 KB)
[v2] Mon, 9 Feb 2026 21:18:55 UTC (15 KB)
[v3] Wed, 7 Oct 2026 15:18:06 UTC (14 KB)