vLLM 跨实例图像缓存串号漏洞及修复

Hao AI Lab· @haoailab · X·2026-11-14 03:13· 35 分钟前AI 评分48

AI 导读

vLLM 在多服务器共享外部 KV 缓存时存在跨实例图像缓存串号问题：当 prefix cache 与图像处理缓存均关闭时，每台服务器的图像标签从 1 开始计数，导致不同图像在共享缓存中撞上同一 key，第二台服务器会命中并返回另一张图像的缓存数据，且表现为正常缓存命中、不报错。修复方式是让每台服务器在图像标签中加入自己的随机部分，使跨服务器标签不再相同，相关 PR 为 #67461。

正文 · 原文

(6/7) Cross-instance mix-up: a question about one image is answered from another image's cached data.

This breaks content-based caching: two different images must never share a cache key.

Normally, vLLM identifies each image by a content hash. But when both the prefix cache and image processing cache are off, it uses a counter starting at 1 on every server. Since that label still enters an external cache store key, two vLLM servers sharing the store give different images the same key. So when the second server is asked about its own image, it can find the first server's cached data under that key and answer from that other image, with a normal cache hit and no error.

Across two servers sharing an external KV cache store, the second answered from another image's cached data. Changing image labeling fixes this: each server adds its own random part, so labels across servers no longer match.

PR: github.com/vllm-project/vllm/…

来源：Hao AI Lab · x.com