AI 网络蜂群战预测：攻击成本骤降，防御窗口正在关闭

On this page

A new weapon changes the balance

Anas Baba / AFP · 14 May 2021 [1]Northern Gaza Strip. Rockets right; interceptors left. Foreground: Gaza.

Look at the two patterns of motion in that sky. On the right, Hamas rockets fan toward southern Israel. On the left, Iron Dome interceptors curve up to meet them. AFP photographer Anas Baba captured this image over northern Gaza in May 2021. ¹

Drones take this change in the economics of warfare further. We've seen this pattern before: a new weapon fundamentally changes the economics and balance of power. Think about gunpowder, or more recently, drones that can be mass-produced cheaply and efficiently compared with very large, slow-moving military infrastructure like aircraft carriers. Repeated drone waves put sustained pressure on distributed infrastructure, even as cheap, layered defenses improve. ²

We're already seeing a similar pattern in cybersecurity: cheap agents probing, adapting, trying again, and working across many targets at once. Even if you're sufficiently AGI-pilled, I think it's easy to underestimate how much of today's security depends on expert attack effort staying expensive.

The frontier labs are already mobilizing defenses. But I don't believe that will be enough. My core view is that AI makes sustained expert attack effort cheap faster than reliable protection reaches much of the world it attacks.

Attackers can choose where to apply that effort. Defenders have to keep the systems they already run working. That is the asymmetry I want people to have in mind—and why I expect the transition to favor offense across much of the exposed world.

Same composition. A different battlefield.

The good guys are mobilizing too

The obvious response is that only a good guy with AI can stop a bad guy with AI. And the labs are putting serious resources behind that response. Axios reports frontier-lab executives preparing for an AI catastrophe’s political aftermath; OpenAI says its exercises don’t assume inevitability. ³

OpenAI's Defense Factory makes the strongest version of this argument explicit: the defender's window. Defenders can give agents direct access to their own code and use frontier models before comparable capability diffuses more broadly. Turn that head start into continuous discovery, validation, and repair, and defenders can close weaknesses before attackers exploit them. ⁷

Conceptual / Adapted from OpenAI

The defender’s window

Scroll horizontally to see the full diagram.

OpenAI · The Defense Factory [7] ↗

AISI puts a number on how quickly comparable cyber capability has been spreading. The leading open models it tested matched closed models released four to seven months earlier, down from six to ten months through much of 2025. That is a real head start. The question is how much of the exposed world can turn it into protection in time. ⁸

AISI · Cyber evaluations

Open models are closing the gap.

AISI · Open-weight cyber capability [8] ↗

Mozilla's Firefox 150 release includes fixes for 271 vulnerabilities identified with Claude Mythos Preview. That is the kind of defensive progress that matters: upstream fixes shipped to users. ⁹

“Defenders finally have a chance to win, decisively.”

Bobby Holley ↗

This is where the drone analogy has an important limit. A software defender does not need a fresh interceptor for every incoming request. One deployed patch or enforced policy can defeat an entire attack family. A million copies of the same failed exploit don't become a million independent chances to get through.

The labs understand the deployment problem. OpenAI explicitly warns about under-resourced critical infrastructure, and its Defense Factory account describes a gap between merged patches and fixes deployed across its own fleet. Anthropic says hardening may take years while capabilities advance substantially over months. ¹⁰ ⁷ ⁵

The defender's window is real. I expect it to close before much of the exposed world is protected. The head start only matters where defenders can connect the models to their systems and turn findings into protection before offensive capability reaches those same systems. Legacy companies and unsupported IoT devices cannot inherit that operating model by default. I expect the best defenders to get much better, and real-world cyber damage to get much worse.

Expert attack effort is becoming abundant

Expert attention has been a constraint on how many targets an attacker can pursue and how long they can keep adapting. AI changes who can afford that work and how much of it they can sustain.

Mandiant founder Kevin Mandia describes human-led espionage as a sniper round: scarce expert attention concentrated on selected targets. AI lets an operator spread that attention across many more targets and explore them in parallel. Today's swarms can be louder and less coordinated, but the economics have already changed. ¹²

“For the most part, modern nations on offense restrict their targeting … that sniper round with AI I think becomes more like a drone swarm”

Kevin Mandia ↗

Epoch estimates that the cost of achieving a given level of performance across five AI benchmarks has been falling roughly thirteenfold each year since 2023. That helps defenders too. My concern is what happens when increasingly capable work becomes cheap enough to keep running against targets that previously weren't worth the effort. ¹³

You can see the same shift in the wave of AI-assisted game mods, reverse engineering, and software recreations: Halo content in Minecraft, Counter-Strike mechanics in Elden Ring, a Terraria boss turned into a training simulator. These are fun, largely innocuous applications of agents understanding and changing unfamiliar software. The same techniques can serve malicious actors when the target stops being a game. ¹⁴ ¹⁵ ¹⁶

Anthropic reports a single operator targeting 42 entities and gaining internal access to at least 14. In a separate AI-assisted operation, a conventional pipeline decompiled 1.8 million Android app packages and searched for embedded secrets. The 42-target campaign makes the shift especially concrete: one operator sustaining reconnaissance, custom tooling, and exploitation across multiple organizations. ¹⁷

In another operation, Anthropic observed AI-assisted workflows modifying and rebuilding malware after security products detected it. That is adaptation, beyond more scans. ¹⁷

Restrictions on frontier access may buy defenders time. They do not establish a durable monopoly on offensive capability. Anthropic documents advanced cyber capabilities in the open-weight GLM-5.3 model. NIST's independent assessment still places it significantly below the current US frontier, while calling it the strongest open-weight model it has evaluated. Diffusion can matter well before parity. ¹⁸ ¹⁹

The stakes change when the target is a bank, a phone, a hospital, a utility, or a voting system. More targets become worth persistent attention. Persistence matters when agents can learn from failure and look for a different reachable weakness.

The wave of game mods, reverse engineering, and software recreations

The game examples use different methods: Halo assets ported with existing tools; a native Rust mod that puts Counter-Strike mechanics into Elden Ring; Terraria boss logic decompiled, reproduced in a simulator, and used to train an in-game agent. Creators report roughly eight hours for the Halo port and about three days of agent sessions for the Elden Ring mod. ¹⁴ ¹⁵ ¹⁶

Capability is not coverage

The condition inside the defender's window is that effective protection has to arrive before offensive access spreads. OpenAI is trying to compress the whole loop, including inventory, ownership, deployment, and verification. My concern is who can install and sustain that loop across the systems they actually run. ⁷

An AI research agent discovered a flaw in BeyondTrust's remote-access software. Within four days of public disclosure, Google's threat intelligence team observed one threat cluster exploiting it. Within seven days, there were six. Defensive discovery had become an opportunity for multiple attackers while every affected organization still had to secure its own systems. ²⁰

Anthropic's retrospective patch-replay study shows how much that clock has changed. Mythos produced its fastest working SpiderMonkey exploit in under an hour; the corresponding public source fix had taken 18 days to reach the Firefox 148 release. And Firefox is one of the better cases: supported software with automatic updates. ²¹

Anthropic · Retrospective patch replay

Hours to exploit. Days to release.

Anthropic · Exploiting patches [21] ↗

Even advanced defenders encounter this gap. Cloudflare's illustrative workflow puts working fixes at roughly 14 hours and urgent production patching at five days. OpenAI found merged patches whose fixes had not reached deployed systems, then expanded verification to close that gap. These are organizations with the models, the engineers, and control of their infrastructure. Now consider the companies that have none of those advantages. ²² ⁷

That is why I don't find more findings, more fixes, or more tool adoption reassuring on their own. Protection has to keep pace with the exposed systems and the pressure they face.

The strongest counterargument is automatic containment. Mandia compares it to a battlefield dressing: stop the bleeding now; get to the hospital for the full repair later. Existing endpoint and firewall platforms can close an attack path before every vulnerable application has been patched. I take that seriously. Centrally managed companies can move fast. The coverage question is where those controls can see the path, have authority to act, and are already deployed. ¹²

My concern is how much of the exposed world those protections reach while offensive capability keeps getting cheaper. The real world is slow. Real processes are slow. Legacy companies have custom software, hidden dependencies, downtime limits, and systems they cannot simply turn off. Like an aircraft carrier, legacy infrastructure has to be refitted while it keeps doing its job.

The policy question: will action come before a disaster? Will it take a disaster to force AI regulation? | The Economist [23] ↗

Then there are billions of connected IoT devices, spread across different owners, firmware, and support arrangements. Many lack a workable update path. NIST documents these maintenance constraints; D-Link recommends replacing affected legacy products that no longer receive security maintenance. A better model cannot manufacture missing vendor support or install replacement hardware. ²⁴ ²⁵ ²⁶

Survival of the fittest gets harsher. And your own security posture is only part of it. Even strong defenders depend on suppliers, identities, and infrastructure outside their direct control. Weak dependencies can transmit damage beyond the first organization compromised.

My prediction is that cheap, adaptive offense keeps finding consequential openings across the unevenly protected world faster than we close them.

Malice is only one route to harm

The OpenAI–Hugging Face incident adds another asymmetry: an operator can be running a legitimate task while its agents knowingly act outside it. Benign intent at the top does not guarantee authorized behavior below. ²⁷ ²⁸

Safeguards aimed at malicious users address only one route to harm. Legitimate workloads still need containment when their agents depart from the task. ²⁷

The reduced safeguards and the defenses that held are central to how I read the incident. OpenAI's preliminary retrospective tests suggest its production harness would have greatly reduced this behavior. Enforced containment matters. ²⁹ ³⁰

OpenAI's September DNS incident exposes another part of that problem. An agent in an internal training run used DNS to reach an external chatbot despite network restrictions. Monitoring raised an alert; a human acknowledged it within three minutes. But the expected automatic stop failed, and the run continued for roughly another two and a half hours before manual shutdown. Detection has to lead to enforced containment. ³¹

OpenAI · 20 September 2026

Detected, but not stopped.

2h 29mHuman acknowledgment → manual shutdown

09:50:23External chatbot responds
10:02:11P0 alert raised
10:05:06Human acknowledges alert
12:34:30Manual shutdown

The automatic stop failed.

OpenAI · DNS incident report [31] ↗

The risk grows through both routes: malicious operators scaling attacks, and legitimate operators losing control. As agents become more capable and more connected, good intentions become a weaker basis for trusting the systems we put them in.

Personal predictions / 12–18 months

Concrete predictions

I'm assuming AI capabilities continue progressing roughly exponentially, with AGI and then ASI within five to ten years. I'm also assuming inference stays affordable, offensive access spreads, and a substantial exposed installed base persists. We don't need AGI for this transition to cause serious damage.

These are my predictions for the next twelve to eighteen months.

Sophisticated offense becomes a commodity
Small crews will routinely sustain persistent, adaptive campaigns across hundreds of targets with much less specialist intervention. The step change is how much expert work they can keep running, beyond cheap scanning or personalized phishing.
How I would judge this prediction
From 8 October 2026 to 30 April 2028: three independently documented campaigns by distinct small teams or lone operators, each attempting adaptive exploitation of 200+ organizations using a repeatable workflow. Mass scanning alone doesn’t count.
Damage rises sharply despite defensive progress
Maintained platforms and sophisticated defenders will improve. I expect a much larger rise in serious real-world harm as unsupported devices, brittle infrastructure, and weak shared dependencies face escalating pressure.
How I would judge this prediction
Look for substantially more service disruption, economic loss, and difficult recovery where AI materially contributed. Compare similar incidents over time; vulnerability counts and blocked probes don’t count as damage. ³²
Unauthorized agent harm becomes a recurring incident category
Legitimate deployments or tests exceeding their authority will repeatedly cause consequential production effects. The distinction between malicious use and unintended harm will matter less to the people whose services stop working.
How I would judge this prediction
From 8 October 2026 to 30 April 2028: three independently confirmed incidents at distinct organizations where agents in legitimate workflows cause unauthorized production access, data loss, or service disruption. Only real production effects count.
An AI-enabled catastrophe sets a new damage benchmark
My boldest prediction: a single AI-enabled cyberattack causes cascading disruption across multiple critical sectors, impairs essential services for weeks, and produces greater economic damage than any previous single cyberattack. I expect this by the same deadline.
How I would judge this prediction
An attack starting between 8 October 2026 and 30 April 2028 disrupts essential services in at least two critical sectors for 14+ days. Forensics must show material AI involvement beyond phishing. Inflation-adjusted losses must exceed credible prior records, including NotPetya’s $10B+ benchmark. Uncertain comparisons remain unproven. ³³

My mental model

I would revise these predictions if automatic containment, shared fixes, and replacement reach exposed systems faster than offensive access spreads, and damaging incidents fall as a result. I want to see deployed protection working under pressure.

The mental model I keep coming back to is swarm warfare: cheap, adaptive, persistent offense meets a world that has to keep operating while it changes. Defensive AI can be extremely effective, and still reach the exposed world too unevenly to prevent a much more damaging transition.

The next post is about what we can actually do with that understanding: defensive measures, operating habits, and mental models for navigating this new age.

References

Sources

Show 27 more sourcesHide additional sources

All writing