arXiv cs.AI· Mattias D\"urrmeier, Alberto Lerner·· 12 小时前AI 评分34
从数据系统视角看 LLM 长上下文本地推理的瓶颈
Local Inference of LLMs on Long Contexts: A Data Systems Perspective
AI 导读
一项跨多代 GPU 的研究分析了 LLM 长上下文本地推理的瓶颈，发现自注意力主导运行时间，且当前自注意力结构使关键 GPU 资源显著未被充分利用，最新 GPU 往往表现最差。研究将这些瓶颈归为数据管理问题，涉及 KV cache 分页、缓存利用率与 GPU 内同步，并向数据系统社区提出后续研究方向。
正文 · 原文
Abstract:Large language models can solve complex problems across a variety of subjects, such as mathematics and coding. For such complex analytical tasks, models with a long context perform better. Serving models with long context is currently mostly done on powerful systems with multiple GPUs, which are too expensive for most users. An alternative is local inference, or serving a model on a user's own machine, which removes per-token costs and rate limits, besides keeping sensitive data private and giving users full control over the models they run. The common understanding, however, is that local LLMs need compression (e.g., quantization) to run on a single GPU. In this paper, we set out to study the limitations that prevent serving long-context models for single users, besides the capacity issue. Rather than characterizing a single model on one GPU, we analyze how different models behave across several GPU generations. This lets us uncover not just individual bottlenecks but broader trends in where inefficiencies arise. Our analysis confirms that, as expected, self-attention dominates the runtime for long-context inference. More unexpectedly, our evaluation reveals that the current structure of self-attention leaves key GPU resources significantly underutilized, and the newest GPU is often the least well served. Many of these findings are fundamentally data management problems, from paging the KV cache to improving cache utilization and synchronization in the GPU, which is why we bring this study to the Data Systems community. To engage this community, we present our findings as a series of experimental observations and formulate recommendations for future work.
| Subjects: | Databases (cs.DB); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.28223 [cs.DB] |
| (or arXiv:2610.28223v1 [cs.DB] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28223 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Alberto Lerner [view email]
[v1]
Tue, 20 Oct 2026 13:56:12 UTC (292 KB)
来源：arXiv cs.AI · arxiv.org