SGLang DP 注意力下取消请求丢失问题
SGLang 在 DP 注意力下存在取消请求丢失问题:快速取消可能先于对应的生成请求到达而被丢弃,导致请求一直运行到 max_new_tokens。原因是默认 DP 控制路由下生成请求直达所属 rank,而 abort 先到 rank 0 再广播,两条路径无顺序保证。
Hao AI Lab· @haoailab · X·2026-11-18 02:27· 53 分钟前AI 评分40
AI 导读
SGLang 在 DP 注意力下存在取消请求丢失问题:快速取消可能先于其对应的生成请求到达而被丢弃,导致请求一直运行到 max_new_tokens。原因是默认 DP 控制路由下生成请求直达所属 rank,而 abort 先到 rank 0 再广播,两条路径无顺序保证。
正文 · 原文
(6/8) Lost cancellation: under DP attention, a rapid cancel can overtake its own generation request and be dropped, leaving the request running to max_new_tokens.
This breaks cancellation finality: once an abort request succeeds, the cancelled request must not run to completion.
With the default DP control routing, a generation request goes directly to its owning rank while an abort goes to rank 0 and is broadcast across ranks, with no ordering guarantee between the two paths. If the abort arrives first, the scheduler has not seen the request yet, drops the abort, and runs the request to its token limit on the next poll.
On GB200, 33 of 204 aborts sent within 20 ms were lost and ran to max_new_tokens, dropping to 0 of 204 once aborts travel in order on each rank's work socket.
PR: github.com/sgl-project/sglang…
来源:Hao AI Lab · x.com
来源:RadarAI · aihot.news