arXiv cs.AI· Yile Gu, Zhen Zhang, Shaowei Zhu, Xinwei Fu, Jun Wu, Yida Wang, Baris Kasikci·· 11 小时前AI 评分34
Ekka：自动诊断 LLM 推理中的静默错误
Ekka: Automated Diagnosis of Silent Errors in LLM Inference
AI 导读
Ekka 是一个自动诊断 LLM 推理静默错误的系统，通过将目标框架与语义正确的参考框架的中间执行状态对齐比较来定位根因，在真实静默错误基准上达到 80% pass@1 和 88% pass@5 诊断准确率，优于现有系统。它还在服务框架中新诊断出 4 个静默错误，均已获开发者确认。该工作入选 ICML 2026 并已开源。
正文 · 原文
Abstract:LLM serving frameworks are quickly evolving with a complex software stack and a vast number of optimizations. The rapid development process can introduce silent errors where output quality silently degrades without any explicit error signals. Diagnosing silent errors is notoriously difficult due to the substantial semantic gap between the high-level symptoms and the low-level root causes. We observe that diagnosis of silent errors can be effectively framed as a differential debugging problem by leveraging the existence of semantically correct reference implementations. We propose Ekka, an automated diagnosis system that identifies root causes by systematically aligning and comparing intermediate execution states between a target and a reference framework. We constructed a benchmark of real-world silent errors from popular serving frameworks, where Ekka shows 80% pass@1 diagnosis accuracy and 88% pass@5 diagnosis accuracy, outperforming state-of-the-art systems. Ekka also diagnoses 4 new silent errors from serving frameworks, all of which have been confirmed by the developers.
| Comments: | ICML 2026, open-source at this https URL |
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Software Engineering (cs.SE) |
| ACM classes: | D.4.5; I.2.1 |
| Cite as: | arXiv:2606.04594 [cs.DC] |
| (or arXiv:2606.04594v2 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2606.04594 arXiv-issued DOI via DataCite |
Submission history
From: Yile Gu [view email]
[v1]
Wed, 3 Jun 2026 08:32:13 UTC (1,140 KB)
[v2]
Tue, 3 Nov 2026 23:54:41 UTC (1,140 KB)
来源：arXiv cs.AI · arxiv.org