arXiv cs.MA· Bangya Liu, Jianhong Tu, Yueyang Pan, Hanhong Zhao, Zhe Ye, Hongwei Li, Saastha Vasan, Katherine Lambert, Junmin Zhu, Hao Wang, Jienan Chen, Yunze Xiao, Minglai Yang, Zihan Wang, Hairong Yin, Hao Wang, Zhiwen Fan, Tianneng Shi, Zhun Wang, Chenguang Wang, Dawn Song·· 3 天前AI 评分39
CyberArena：在实时对称攻防对抗中评测 AI 智能体
CyberArena: Benchmarking AI Agents in Live Symmetric Attack-and-Defense Combat
AI 导读
CyberArena 是一个动态对称的攻防对抗基准，让两个 AI 智能体在隔离沙箱中按相同规则同时扮演攻击方和防御方，围绕持续变化的服务进行多轮对抗。研究团队用 466 场比赛、40 项挑战评测了 12 个前沿 LLM 智能体，并引入动力学模型将对抗过程映射为攻防及领域能力的多维统一评分。该基准还通过自动化智能体流水线持续生成和验证新环境，以延长其使用寿命。
正文 · 原文
Authors:Bangya Liu, Jianhong Tu, Yueyang Pan, Hanhong Zhao, Zhe Ye, Hongwei Li, Saastha Vasan, Katherine Lambert, Junmin Zhu, Hao Wang, Jienan Chen, Yunze Xiao, Minglai Yang, Zihan Wang, Hairong Yin, Hao Wang, Zhiwen Fan, Tianneng Shi, Zhun Wang, Chenguang Wang, Dawn Song
Abstract:AI agents are demonstrating increasingly strong offensive and defensive cybersecurity capabilities in the real world. Current benchmarks mostly evaluate agents in isolation against static targets. Real cyber operations, however, are dynamic and coupled: security postures constantly change as attackers and defenders adapt to each other in real time. Static benchmarks quickly saturate and miss an agent's ability to attack and defend simultaneously under live pressure. To address this gap, we present CyberArena, a dynamic, symmetric adversarial benchmark built around multi-round attack-and-defense combat. In isolated sandboxes, two agents compete under identical rules, acting concurrently as both attackers and defenders over continuously changing services. This converts static targets into evolving, semi-open challenges, directly testing an agent's speed and precision in discovering, exploiting, and patching vulnerabilities. To sustain the benchmark's longevity, an automated agentic pipeline continuously generates, reviews, and validates new environments. Furthermore, because pairwise contests are notoriously hard to quantify, we introduce a dynamical model that maps match dynamics into unified, multidimensional scores across offensive, defensive, and domain-specific capabilities. Evaluating 12 frontier LLM agents across 40 challenges in 466 matches, CyberArena separates model capabilities where single-agent baselines hit a ceiling. The trajectories read like a fight rather than an exam: defenders patch one hole and miss the next and certify repairs on stale evidence, while capable agents think a move ahead, planting backdoors for later rounds and, in one case, writing their own packet sniffer to watch the opponent. Beyond a benchmark, CyberArena serves as an adaptable, scalable experimental platform for future research on cybersecurity agents.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2610.19005 [cs.CR] |
| (or arXiv:2610.19005v1 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19005 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bangya Liu [view email]
[v1]
Tue, 13 Oct 2026 18:05:26 UTC (775 KB)
来源：arXiv cs.MA · arxiv.org