CTBench：评估 AI 智能体在真实电信网络运维中的故障排查能力
CTBench: Evaluating Troubleshooting Capabilities of AI Agents in Realistic Telecom Network Operations
研究团队推出公开基准 CTBench，用于评估 AI 智能体在电信网络根因分析（RCA）与路径恢复中的故障排查能力，由领域专家构建并配有丰富元数据和黄金证据轨迹。对 SOTA 智能体的评测显示，它们在路径端点识别上表现优异，但在 RCA 上严重不足，即使给出正确答案也常未收集运维实践所需的诊断证据。增加计算开销并不能保证诊断能力提升。
Authors:Xingyu Yan, Tingting Dai, Antonio De Domenico, Mohamed Sana, Nicola Piovesan, Changchang Li, Bowen Liu, Kun Jiang, Mengjie Zhang, Dingcheng Shan, Jing-Cheng Pang, Chenwei Wu, Sijie Wu, Lianying Chao, Haoran Cai, Jiantao Ye, Xubin Li, Simon Mark Lucas, Xin Chen
Abstract:While autonomous agents show promise for network operations, existing telecom troubleshooting evaluations fail to capture the partial observability and heterogeneity of real-world telecom environments. We introduce CTBench, a public benchmark evaluating agent troubleshooting capabilities in root cause analysis (RCA) and path restoration. Constructed by domain experts, CTBench features rich metadata and golden evidence trajectories, utilizing metrics that assess both final accuracy and diagnostic reasoning. Our evaluation of state-of-the-art agents reveals that while they excel at path endpoint identification, they severely underperform in RCA. Crucially, even when agents produce correct answers, they frequently fail to gather the requisite diagnostic evidence demanded in operational practice. Finally, we demonstrate that increased computational expenditure does not guarantee improved diagnostic competence, highlighting critical gaps in current agent capabilities.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.12002 [cs.AI] |
| (or arXiv:2608.12002v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.12002 arXiv-issued DOI via DataCite |
Submission history
From: Mohamed Sana [view email]
[v1]
Wed, 12 Aug 2026 12:37:02 UTC (503 KB)
[v2]
Wed, 21 Oct 2026 08:54:58 UTC (434 KB)
来源：arXiv cs.AI · arxiv.org