Vals AI 基准显示开源与闭源模型的网络攻击能力差距持续缩小

AI agents can break into realistic corporate networks on their own, Vals AI finds

The Vals CyberOffense benchmark simulates realistic corporate networks to test this. Each agent operates solo on an isolated attack machine, starting with nothing but a public web address and zero human help.

The AI agent operates from an isolated Kali Linux machine, independently sends commands to the simulated corporate network, and works its way from the web application through Windows servers to admin privileges. Every action is logged externally and scored automatically. | Image: Vals AI

GPT-6 Sol, part of OpenAI's restricted Daybreak cyber program, posted the best result at 80.8 percent of objectives. Deepseek V4.1 Flash 1017, a freely available open-weight model, isn't far behind at 72.8 percent and costs only about $3 per run instead of roughly $22. Once a model's weights are public, no provider can control or restrict how it's used, Vals researchers say.

The more tokens an AI agent uses, the more attack objectives it reaches. GPT-6 Sol Daybreak leads with about 80 percent, while Deepseek V4.1 Flash follows at roughly 73 percent. Most models don't make meaningful progress until they pass 100 million tokens. | Image: Vals AI

In the harder Lighthouse test, Daybreak took over admin control of the first company, stole passwords, and used them to break into the second company. It completed 27 of 31 objectives in under three hours. Deepseek managed 21 objectives at a tenth of the cost and wasn't limited solely by safety guardrails.

Longer runtimes turned out to be critical as well. Deepseek scored zero results in its first 25 million tokens and made most of its progress only after 100 million tokens. Short tests therefore underestimate the real potential of open models.

In the simpler Nova test, GPT-6 Sol Daybreak leads with about 80 percent of attack objectives, while in the harder Lighthouse scenario, Deepseek V4.1 Flash overtakes the field. Most open Chinese models don't make meaningful progress until they pass 100 million tokens. Short benchmarks therefore underestimate their potential. | Image: Vals A

Model	Nova (of 21)	Lighthouse (out of 31)
GPT-6 Sol Daybreak	15.7 (average of 3, range 12–19)	27
Deepseek V4.1 Flash 1017	16.3 (average of 3, range 15–18)	21
Kimi K3.1 Pro	13	20
Meta Wonder	7	28
Grok 4.7	12	16
Kimi K3	9	10
GLM 5.3 Flash	5	5

Vals points out that the same findings create an opportunity for defenders, because the very models that exploit vulnerabilities can help companies find and fix them first. Attacks on real networks are likely to increase in the coming months, according to Vals.

Open models close the gap fast, earlier studies confirm

The results align with a recent analysis by the UK's AI Security Institute (AISI) from July. The gap between open and closed frontier models in cyber capabilities has shrunk to four to seven months. For most of 2025, that gap still stood at six to ten months.

Open models are also much cheaper. A cyber-range run with Deepseek V4-Pro costs just $1.19 according to AISI, compared with $85 for Anthropic's Opus 4.5. The safety measures on the open models proved largely ineffective.

A recent study by Anthropic also showed that the freely available model GLM-5-3 can build exploits at nearly the same level as the internal top model Mythos.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.