Autonomous Is Not Autonomy

Part 3: Worms, the self-terminating attack, and the only scenario that works

Series: Part 1 · Part 2 · Part 3

This is the last of three parts. Part 1 made the dependence argument and the botnet arithmetic; Part 2 measured capability and took the Hugging Face incident apart. This part asks what propagation would cost, why a grid attack is self-terminating for the attacker on that grid, what defenders are doing, and what the only scenario that works looks like. The appendix and the revision record are here.

Contents of this part

7. Worms, and why an AI agent is not one
  7.1 The comparison
  7.2 Two architectures, and they fail in opposite directions
  7.3 The concession, and it matters
  7.4 The hardest objection: what if there is no switch at all?
8. The self-terminating attack
  8.1 Twelve hours, not days
  8.2 But fuel is not the binding constraint. Cooling is.
  8.3 And the corollary is the sharpest thing in this article
  8.4 The last mile goes first, and there is no mandate holding it up
  8.5 The recursion
9. What people are actually doing about it
  9.1 The banks are blocking AI agents, for gloriously boring reasons
  9.2 The 2023 ChatGPT bans were a compliance pause, and the system absorbed it
  9.3 AI watching AI is now a product category, and it has been measured against the real thing
  9.4 The access gate is the real control, and the vendors built it themselves
  9.5 What a government lab changed after its own incident
10. The only scenario that works
  10.1 Five state ICS programs, and what each one actually did
  10.2 A detail from the Sandworm row that is worth the whole section
  10.3 Volt Typhoon is the real template, and it shows the cost
  10.4 The geographic bound is in the primary source
  10.5 Why "global" fails specifically
  10.6 The amendment: NotPetya, and it mostly confirms the claim

And so

Revisions

Appendix A: How inference actually works
  A.1 Four facts
  A.2 What an "expert" is, and why mixture-of-experts doesn't save you
  A.3 Where the numbers in this article come from

7. Worms, and why an AI agent is not one

The apocalypse story needs propagation. A model that can only run where someone installed it is a tool; a model that spreads is an adversary. And we have a mental model for software that spreads, because we lived through it.

So it is worth asking what a worm actually is, because the answer turns out to be the opposite of what the analogy implies.

A worm is self-replicating code that propagates without user action. It needs exactly three things: a propagation path, which is an exploitable service or an abusable trust relationship; a target discovery mechanism, which is almost always scanning; and a payload carrier, either self-contained or a second-channel fetch over TFTP, FTP, HTTP or SMB.

Now the fact that matters, and it is genuinely counter-intuitive.

Worms were devastating because they were stupid.

Slammer was 376 bytes. The entire worm fit in a single UDP packet. It had one exploit and a random IP address generator. It did not reason about its target, adapt to what it found, or make decisions. And not thinking is precisely why it crossed the internet in fifteen minutes. There was nothing to wait for.

7.1 The comparison

7.2 Two architectures, and they fail in opposite directions

There are only two ways to build a propagating AI agent.

Weights as payload. Copy the model to each host and run it there. This is what Anthropic warned about in October 2026, and it is what the one published AI-worm prototype does. Guan, Blanchard, Foerster, Jia, Huang and Papernot built a worm that "uses compromised machines to run open-weight large language models" and tiers them, so that "each compromised GPU-equipped node provides reasoning for lightweight agents on downstream devices," across a test network of Linux, Windows and IoT machines (AI Agents Enable Adaptive Computer Worms, arXiv:2606.03811, 2 June 2026; project page). Two things about it matter here. It ran "exclusively in a contained virtual network with hypervisor-enforced isolation," and the authors "intentionally omitted implementing any standard malware capabilities that complicate detection or removal." So it demonstrates the architecture, not the architecture surviving contact with a defender. It is genuinely autonomous once resident, and it is not hypothetical. In Anthropic's eight-node cloud test, one open-weight agent model, GLM-5.2, carried out the replication steps and stood up Llama-3.1-8B as the served model on all eight nodes in four of five trials. Separately, they ran a 26-billion-parameter mixture-of-experts model with distributed inference across a fleet of fifteen smart TV boxes. Section 7.4 goes through both, and through what they do not show.

But it is a fat worm. Gigabytes per hop instead of bytes. Anthropic's own image for the result is a "digital slime mold," slow and distributed, and their arithmetic puts a fleet of TV boxes the size of Kimwolf, which they report at over two million devices on third-party figures, at about forty H100s' worth of inference in total. And their conclusion is worth quoting because it is appropriately careful: "We don't yet have a full rogue replicator," with the individual steps described as increasingly within reach.

Key as payload. Carry a credential or a URL, and do the thinking somewhere else. This is LLMjacking, and Sysdig's June 2026 observation of a complete automated offensive pipeline running against a single misconfigured Ollama server is the worked example. The payload is tiny and propagation is fast.

And it is a thin client for a computer somebody else owns. Permanent network dependency, and a revocable control point.

You cannot have both. Autonomy costs payload size and speed. Speed costs autonomy.

A self-sufficient AI worm is slow and conspicuous. A fast, light one is a remote control for a machine that someone else can switch off. This is the dependence thesis from Section 1.3 (Part 1), restated as an engineering trade-off rather than an assertion, which is the strongest form it can take.

7.3 The concession, and it matters

Against the same vulnerability, an AI worm is strictly worse than a dumb worm. Slower, larger, costlier per host, less reliable. If there is one exploit that works everywhere, reasoning is pure overhead.

The AI only earns its cost where the target environment is heterogeneous, where no single exploit works everywhere and the attacker genuinely has to think.

But that is exactly the case where worm speed is unavailable anyway. Heterogeneous targets require per-target tailoring, and per-target tailoring is slow by definition. So "worm-speed global AI attack" is a contradiction in terms: the conditions that make AI useful are the conditions that make worm propagation impossible.

Which is also, as Section 10 will show, exactly why the most capable state actor currently inside American critical infrastructure uses no malware at all.

7.4 The hardest objection: what if there is no switch at all?

Everything in this article has assumed that a model runs somewhere reachable. Section 1 (Part 1) called that somebody else's hardware: still owned, still drawing power, still findable. Section 7.2 split propagating agents into two architectures and argued that both of them keep a dependency, either on a datacenter you can raid or on a credential you can revoke.

There is a third possibility, and the people who demonstrated it are not trying to sell you anything.

On 12 October 2026 Anthropic's Frontier Red Team published work on whether small models can run on cheap, weakly secured consumer hardware (AI models may be capable of self-replicating without access to a datacenter). They sharded Gemma4-26B-A4B, a 26-billion-parameter mixture-of-experts model with four billion active, across fifteen off-brand Android TV boxes at roughly $55 each, with four gigabytes of RAM apiece, partitioned by layers using pipeline parallelism, at 0.39 tokens per second, falling to about 0.3 over a 32,000-token context.

Be precise about what that was, because the result will be cited loosely. It was one of three separate experiments, not one end-to-end demonstration. The TV boxes ran inference. A separate test on a firewalled network of eight cloud nodes ran the replication steps, with simplified exploitation, and the agent model that completed them, GLM-5.2, spread to all eight nodes in four of five trials. A third, scripted-setup run covered seventeen nodes. Nobody chained those into a worm that spreads across real consumer devices and then runs on them. Anthropic says so plainly, "we don't yet have a full rogue replicator," describes the cloud result as an upper bound on real-world success, and gives its forecast that small models could reach the threshold by the end of 2026 as an extrapolation from a capability trend rather than a measurement. That is the right way to publish a forecast and the wrong thing to quote as a finding.

Their framing of the consequence is the problem:

Our experiments suggest that small AI models may be months away from being capable enough to self-replicate on commercial internet-of-things (IoT) devices, like smart ovens and off-brand Android TV boxes. A replicator like this would be challenging to shut down, since there's no centralized point to shut off—though it may be slow (with a doubling time of days).

Take that seriously, because it defeats something I claimed. The clean version of this article's argument says the attack runs on an account, or a server, or a facility, and that somebody can reach it. A replicator whose weights are already distributed across a hundred thousand consumer devices has no account to suspend, no provider to notify, no exposed port to find and no single owner to serve. Section 1.4 (Part 1) conceded that attacks on hijacked self-hosted inference have no invoice. This concedes more: they may have no address. Or rather, a hundred thousand addresses, each with an owner and a network connection, and no single administrator whose one action disables them all. That is a different problem from having no address, and the difference is this section.

So what is left? Four things, and the last one is the answer, with its limit stated.

First, "no central point" is not a new condition. It is the normal one, and we have thirty years of experience with it. Section 4.10 (Part 1) listed what botnets have historically been used for. Reread that list with this objection in mind. Grum, Zeus, Conficker, Mirai, 3ve, Methbot. None of those had a switch in the sense this objection means. An infected population never does. They had command-and-control infrastructure, which is a kind of center, but the compromised machines themselves were exactly as distributed and exactly as unowned as the architecture now being described. And they were degraded or dismantled anyway, by a slow unglamorous combination of sinkholing, ISP notification, vendor patching, device recalls and law enforcement against the humans running them.

The strongest single data point is already in this article and I did not notice what it was doing there. BadBox 2.0 compromised more than ten million uncertified Android devices on Google's April 2025 count, the figure in its July 2025 complaint. HUMAN Security's Satori team, which exposed the operation in March 2025, counted over a million at the time and called it the largest botnet of infected connected-TV devices ever uncovered. The two numbers are different dates, not a disagreement. That is precisely the hardware class Anthropic bought for fifty-five dollars a unit. It reached the scale the objection requires. And it was identified, named, partially sinkholed, and made the subject of a federal lawsuit and an FBI advisory. It has not been eradicated, and I am not going to pretend it has. What its history falsifies is the claim that a distributed device population cannot be found. We keep finding them. Finding is not cleaning, and the gap between the two is the rest of this section.

Second, the control surface moves rather than disappearing, and the people who raised the alarm also wrote down where it moves to. An infection you have to hunt device by device is hopeless. But the exploitable population is not enumerable one device at a time; it is enumerable by model and firmware version, because the devices that are exploitable are the ones that do not get updates. That turns a takedown problem into something closer to a product recall: a known list of SKUs, a known set of importers, a known distribution chain, and the regulatory machinery that already exists for unsafe consumer electronics.

I did not have to invent that argument, because Anthropic's own paper makes it. Their blue-team section casts the problem as epidemiology, driving the reproduction rate below one so the thing dies out, and it reaches for a precedent that is already in this article: when an ISP quarantined Mirai-infected subscribers behind a cleanup page, it "cleared 92% of infections within 14 days, against 74% with no notification." I went back to the study behind that figure, and it carries less than the headline suggests (Çetin et al., Cleaning Up the Internet of Evil Things, NDSS 2019). It is 220 infected customers at one medium-sized European ISP. The 74 percent who cleaned up with no help at all did so, in the authors' words, "probably reflecting the non-persistent nature of the malware": Mirai lives in memory, and a reboot removes it. The quarantine's own contribution is the eighteen points on top of that, and the authors close by warning that "it will be hard to scale up the walled garden solution because of the weak incentives of the ISPs." A weight-based replicator is a multi-gigabyte payload written to flash. It is persistent by construction, and the 74 percent floor does not exist for it. So the honest reading is that ISP quarantine is a proven mechanism with a measured effect, demonstrated against an easier target, at one ISP that had a reason to try.

Anthropic also points at the supply side: "Manufacturers can shrink the pool of targets, with regulation acting to coordinate. For example, default IoT passwords are now non-compliant in UK markets." That is the recall mechanism, named by the lab that built the demonstration, and it is the one that does not depend on ISP incentives.

Third, running is quiet but spreading is loud, and again this is Anthropic's finding, not mine. A thing that needs months to reach meaningful scale has a months-long detection window, and the spreading is not subtle. Their own words: a weight-based replicator at scale "would leave a network signature, visible as an anomaly in ISP flow-based monitoring systems," because newly infected devices pull "multi-gigabyte model payloads" and distributed inference produces "sustained device-to-device traffic between machines that wouldn't ordinarily communicate at that volume." And the trap is structural: "the replicator trades off between replication speed and stealth: the faster it replicates, the more traffic it produces." The quiet phase is the useless phase. The useful phase is the observable one. That is not a hopeful gloss on their research; it is their research.

Fourth, and this is the answer: today, unpluggability is bought with capability, and Section 7.2 tells you the exchange rate.

Look at what maximum unpluggability actually purchased in the demonstration. A 26-billion-parameter model, across fifteen devices, at 0.39 tokens per second in the demo and roughly 0.3 tokens per second over a long context. Call it fourteen hundred tokens an hour, for a model more than an order of magnitude smaller than the frontier, on hardware somebody bought and networked deliberately rather than compromised at random.

Anthropic ran the arithmetic out to a full botnet and it does not rescue the picture. A fleet the size of Kimwolf, the Android-TV botnet they use as their reference, which they report at somewhere over two million devices on third-party figures rather than their own measurement, with every fifteen boxes running a copy, would be on the order of "40 H100s-worth of inference" across the whole swarm. Their own analogy for the result is a "digital slime mold." Forty accelerators of aggregate throughput, spread across two million machines, is the most expensive and least efficient way to assemble forty accelerators that has ever been proposed.

A worked example, so the speed means something. Take Anthropic's 0.39 tokens per second as given. One fifteen-box replica then produces about 1,400 tokens an hour, 34,000 a day, 12 million a year. The token budgets below are this article's working assumptions, the same ones the published script uses for its operational translation, and they are round numbers rather than measurements: about 2,000 tokens for one agentic reasoning step, about 50,000 for a working day of agent output, and about 10 million for a campaign on the scale of GTG-1002 against one target.

Three things follow. Breadth is not the constraint. A Kimwolf-sized fleet is roughly 133,000 replicas, and they can work 133,000 targets at once, each at this speed. Depth is. An intrusion is an interactive loop, and nothing parallel shortens a single chain, so ten months is the time to one enterprise-scale compromise, not to a grid. The ICS record in Section 10 says a disruptive OT attack costs far more than that: Volt Typhoon spent five years per environment with human operators, and TRITON took a state institute years for one controller family. Scale the same ratio and a replicator at this speed is measured in years per target, generating the device-to-device traffic of point three above for the whole of that time. And token speed is the wrong bound for a patient attacker. Spend the years preparing, pre-position scripts, execute without thinking. Tokens per second bounds the thinking, not the execution, which is the case conceded below. What bounds the patient case instead is whether a 26-billion-parameter model can develop a working attack on industrial controllers at all, and Section 5 (Part 2) has no evidence that anything below the frontier can. So the honest answer to "how long" is a lower bound of years per target from arithmetic alone, and no upper bound, because the capability has not been shown to exist at that size.

That is the most autonomous architecture available, and today it is also the least capable one. On the hardware Anthropic tested, there is no configuration that is both unreachable and fast.

I want to be careful about how far that generalises, because it is an observed trade-off and not a law, and this article's own Section 4.11 (Part 1) shows where it bends. A 30B model fits on one consumer GPU and runs at tokens per second with no network traffic at all. The consumer devices that can do that are gaming PCs and high-memory workstations, which ship today with 24 to 128 gigabytes of fast memory. A population of those, compromised, is still a distributed consumer-device fleet with no single administrator, and it runs a model an order of magnitude more capable than the TV-box demonstration at thirty times the speed. Those machines have owners who notice, operating systems with update channels, and endpoint products watching them, so they are easier to reach than a fifty-five-dollar box that never updates. But they are not both out of reach and useless. They are harder to reach and somewhat useful, and the trade-off is a curve rather than a cliff.

The slope of that curve also moves. Section 5 (Part 2) is about small-model offensive capability rising faster than parameter count predicts, which means the bottom of this table gets more capable every quarter without getting any more reachable. What does not move is the physics underneath: a model that needs more memory than one device has needs a network between devices, and a network between consumer devices is the latency-bound regime of Section 4. The curve bends. It does not disappear.

Which matters because of what the scare narrative specifically requires. Not persistence. Simultaneity. Everywhere at once, faster than anyone can respond. A replicator with doubling times in days and a third of a token per second is the opposite of that on both axes. It is a genuine problem, and it is a slow, grinding, attritional problem of the kind our field is actually equipped for, rather than the sudden coordinated one we keep being told to expect.

What I cannot answer, and will not pretend to. A replicator does not have to be fast if it is patient. Accumulate quietly for two years across ten million devices, pre-position whatever the attack needs, then act in an afternoon with no thinking required. Propagation rate, inference rate and execution simultaneity are three different quantities, and nothing measured above bounds the third. That is Volt Typhoon's shape rather than Skynet's, and Section 10 argues that the gate on that scenario is a decision rather than a capability. But "the gate is a decision" only reassures you if the decision is human, and in this scenario that is exactly what is in question. I do not have a confident answer to the patient case. Nobody does, and I would be suspicious of anyone who claimed otherwise. It is not something Anthropic demonstrated or claimed. It is the counterexample that any argument from slow decoding to impossible simultaneity has to answer, and it should be carried as an open hypothesis rather than as either a demonstrated threat or a dismissed one.

So the honest shape of this article's claim, after the objection, is three tiers rather than one switch:

Revocability decreases as you go down that table. So does capability, today, and for a physical reason rather than an accidental one: more independence means smaller models or slower networks, and nothing on anyone's roadmap changes which direction that trade runs. But "today" is carrying weight in that sentence and I would rather say so than have a reader discover it.

What survives the objection is not a single switch and not an inverse law. It is three control regimes that work at different speeds, a bottom tier that is slow and getting less slow, and a defensive question that is the same in all three: whether the people who own the hardware can find, constrain and recover it faster than the thing running on it can replace it. That is the dependence thesis with the comfortable sentence removed. There is no one switch. There is a race, and the physical dependencies are what make it a race the defender can run.

8. The self-terminating attack

Every version of the apocalypse story ends with the grid going down. It is the load-bearing image, because a grid failure cascades into water, fuel, payments, hospitals and communications.

So let us ask the question the story never asks. What happens to the attacker?

An AI that shuts off the power grid shuts itself off. It is the only adversary in history whose weapon destroys its own weapon.

That sounds like a rhetorical flourish. It is an engineering claim, and the numbers are more specific than I expected.

Scope it before going further, because it is only true as stated for one case. It is true of an AI whose compute sits on the grid it attacks, which is what the everywhere-at-once story requires, and it is the case this section works out. It is not true of compute on another continent's grid attacking this one. But that is not an autonomous AI; it is a state with a tool and a datacenter, and what stands in its way is in Section 10, not here. And it is not true of a system that has already placed its execution logic on the target and no longer needs to think during the attack, which is the patient case Section 7.4 could not close.

8.1 Twelve hours, not days

The intuition is that datacenters have enormous fuel reserves. They do not. The Uptime Institute's own journal is explicit: "The Tier Standard: Topology requires this 12-hour fuel storage minimum for all Tiers at 12 hours of runtime at 'N' load" (Uptime Institute Journal, Data center fuel system design and reliability). Twelve hours. For every tier, including the highest.

Two things make the real figure worse. Redundancy halves the usable number, because the rating is what survives losing a component: an operator claiming forty-eight hours across two twenty-four-hour tanks has twenty-four hours of concurrently maintainable fuel. And the widely repeated table saying Tier III requires seventy-two hours and Tier IV ninety-six is vendor folklore. It contradicts Uptime's own text. What changes at the higher tiers is the resilience of the fuel path, not the number of hours: a Tier III site must hold twelve hours with any component, including the tanks and delivery systems, out of service.

Longer runtimes are common as a design or contract choice. They are not a requirement, and beyond the on-site tank you are depending on tanker trucks, which depend on refineries, pumps and payment systems, all of which depend on the grid you just attacked.

8.2 But fuel is not the binding constraint. Cooling is.

This is the part that surprised me, and it is the strongest version of the argument.

The critical detail comes from Schneider Electric's modeling of cooling outages: "cooling components such as CRAC/CRAH fans, chilled water pumps, and chillers are typically not connected to UPSs" (Schneider/APC, Data Center Temperature Rise During a Cooling System Outage).

The uninterruptible power supply carries the IT load. It does not generally carry the heat removal. So between mains loss and generators picking up and chillers restarting, there is a window in which the equipment is running at full power and nothing is taking the heat away.

Schneider's modeled case breaches the allowable threshold in about three minutes and stays unacceptable until roughly seventeen. Cundall's modeling of a total mains loss found that without mechanical UPS support or thermal buffer vessels, ASHRAE thresholds were "predicted to be breached within seconds," and notes the restart problem: chillers struggle to come back because the return water is already hot (Cundall, Why the seconds after failure matter to data centre cooling).

Now the number that makes this a story about AI specifically, with a caveat I owe you. Schneider's modeled baseline is four kilowatts per rack. A GB200 NVL72 rack draws 132 kilowatts, per HPE's specification for the system, split 115 kW liquid-cooled and 17 kW air-cooled. That is thirty-three times the density of the baseline in the thermal model.

The caveat is that you cannot scale Schneider's three-minute figure by thirty-three and call it a result. Schneider modeled an air-cooled room, where the room's air is the thermal buffer. A liquid-cooled AI rack has a different buffer and different failure dynamics, and the engineering guidance I can find for it points both ways at once.

In the direction that weakens this section: the pumps on the chip-side loop and the coolant distribution unit's controls are usually put on the UPS, precisely because the loop cannot be allowed to stop, so a mains loss alone does not stop the flow. The chillers, which take minutes to restart, are the part that drops, and many designs add a thermal store to cover that gap (h2x Engineering, Data Center Liquid Cooling Design for Direct-to-Chip Racks, 19 October 2026, a design guide rather than a measurement). In the direction that strengthens it: the same guide says a liquid-cooled rack "has seconds rather than minutes when cooling stops," cites Vertiv for a high-power chip reaching its thermal limit in under a second once coolant stops, and cites an ASHRAE liquid-cooling bulletin, via Grundfos, for as little as 23 seconds from a pump failure to processor throttling.

I have not found a published thermal model of a 132 kW rack losing its chillers with the loop still running, and until one exists the defensible statement is qualitative. The heat that has to be removed is thirty-three times larger. The margin once flow stops is seconds. What stands between a mains loss and that margin is a UPS on the pumps, a thermal store, and a chiller restart, all of which are design choices rather than standards. The direction of the dependence is not in doubt. The number of seconds is, and it is set by the size of the thermal store.

The frontier-AI datacenter is the most thermally fragile facility class ever built, and its fragility scales with exactly the thing that makes it capable. Twelve hours of diesel is irrelevant if the cooling drops at 132 kilowatts a rack. How long it can drop for is a number nobody has published.

8.3 And the corollary is the sharpest thing in this article

If you were an attacker optimizing to destroy a datacenter rather than merely inconvenience it, you would not want one long blackout. You would want repeated brief interruptions, because every transfer is a thermal event and restarting chillers against hot return water is the hard part.

Which means the grid attack that best destroys AI compute is the grid attack the AI cannot survive long enough to sustain. The optimal strategy is self-defeating. Not as a matter of ethics or deterrence, but as a matter of thermodynamics.

8.4 The last mile goes first, and there is no mandate holding it up

The attacker also loses its own command and control, because consumer connectivity degrades long before the core does.

The history here is instructive and almost universally misreported, so it is worth getting right from the primary record. After Hurricane Katrina, the FCC's 2007 Katrina Panel Order required backup power "for a minimum of 24 hours for assets located inside central offices" and "reasonably required only 8 hours of backup power for such locations," meaning cell sites, remote switches and digital loop carrier remote terminals.

That rule never took effect. It was contingent on OMB approving the associated information collections under the Paperwork Reduction Act. CTIA petitioned for review, the D.C. Circuit held the cases not ripe and placed them in abeyance pending OMB action (CTIA v. FCC, 530 F.3d 984, D.C. Cir. 2008), OMB then disapproved the collection, the Commission declined to override that disapproval, and on 31 July 2009 the court dismissed the petitions as moot and vacated the rules. The FCC's own later summary: these rules "never took effect and were ultimately vacated by the U.S. Court of Appeals for the District of Columbia Circuit" (FCC 11-55, Notice of Inquiry, ¶13 and nn. 27-29).

So there is no federal backup power requirement for cell sites in the United States. There are voluntary commitments, and one notable state exception: California Decision D.19-08-025 requires seventy-two hours in Tier 2 and Tier 3 High Fire Threat Districts.

Typical cell-site battery reserve runs two to four hours by one estimate, four to eight by others, up to about twelve in some builds, with generators extending that at sites important enough to have them. And a detail from the FCC's own docket deserves repeating, because it suggests even the mandated figure was aspirational: Verizon Wireless conceded in an ex parte filing that it "cannot guarantee that every battery designed to provide 8 hours of backup power will actually do so," since batteries "begin to deteriorate the minute they are installed."

8.5 The recursion

The deeper the grid attack succeeds, the faster the attacker's own substrate goes dark. And the attacker cannot repair a substation, drive a fuel truck, refill a day tank, or walk into a switchyard and close a breaker by hand.

Humans can. This is not hypothetical, and we have the receipts from the only place destructive grid attacks have actually been carried out. In December 2015 in Ukraine, power was restored within roughly three to six hours by field staff driving to substations, switching from automatic to manual operation, and re-closing breakers by hand, while SCADA was still infected. In December 2016, Industroyer took out roughly a fifth of Kyiv for about an hour before Ukrenergo's engineers brought it back.

Which connects to Section 5.4 (Part 2)'s uncomfortable finding from the other direction. In the only defense-aware offensive evaluation on record, every model tested scored zero against a web application firewall, and benign decoy hosts alone broke attack chains that worked in isolation. The places these systems actually fail are not exotic. A commodity web appliance. Network segmentation. A human with a flashlight and a procedure.

Boring, already-deployed, unglamorous defense is where capability stops. That is the finding, and nobody wants to put it on a conference slide.

9. What people are actually doing about it

A reasonable objection to everything above is that it describes a world in which nobody is responding. The opposite is true, and the response is more interesting than the threat, because of why it is happening.

9.1 The banks are blocking AI agents, for gloriously boring reasons

As of October 2026, four major US institutions restrict AI agents from customer accounts. Citigroup does not permit third-party AI agents to log in or transact through its digital banking platforms. JPMorgan Chase will block outside agents attempting to use customers' banking credentials, while continuing to support authorized data sharing through open banking. Bank of America is limiting access while it works out liability. Charles Schwab is blocking unauthorized agents and building connections that provide information without permitting agents to move money.

Jamie Dimon's framing on the Q3 earnings call: "You've got to make sure customers aren't hurt by it," and "You don't want rogue agents, and there could be thousands of them."

Now read the stated reasons, because they are the point. Liability for payments an AI tool authorizes. Protection of low-cost deposit bases against AI assistants that would automatically sweep idle cash to higher-yielding competitors. Ordinary third-party risk management.

The controls that would blunt an autonomous financial attack are being installed for commercial and liability reasons, which means they do not depend on anyone believing in AI risk at all. Robustness through self-interest beats robustness through foresight, and it arrives faster.

9.2 The 2023 ChatGPT bans were a compliance pause, and the system absorbed it

This episode also gets retold as evidence of insider fear, and it was not. JPMorgan restricted employee ChatGPT use around 22 February 2023, and contemporaneous reporting framed it as part of normal controls on third-party software rather than a response to any incident. The three real drivers were data leaving the firm with no contract governing retention or deletion, recordkeeping obligations for broker-dealers and advisers, and the requirement to be able to explain model-driven decisions.

What happened next is the lesson. Confidentiality was largely solved by contract and architecture within about a year, through enterprise agreements and private deployments. No AI-specific recordkeeping rule ever appeared: FINRA's Regulatory Notice 24-09 in June 2024 simply reminded members that existing rules apply, and the SEC's August 2024 off-channel communications sweep charged twenty-six more firms with roughly $393 million in combined penalties. Firms logged prompts and outputs into retained systems and extended existing model-risk frameworks.

The existing regulatory regime absorbed the new technology without being rewritten. That is unglamorous, fast, and almost never reported.

9.3 AI watching AI is now a product category, and it has been measured against the real thing

There is a market: Arthur launched agent discovery with runtime guardrails in December 2025, Fiddler repositioned as an enterprise agent control plane in January 2026, and ServiceNow, Microsoft and OneTrust all ship control planes for agent governance. Consolidation is underway, with Lakera acquired by Check Point and Traceloop by ServiceNow. I would treat the feature claims with the usual caution, since almost all of this is vendor self-description.

But the connective tissue is not a vendor claim. It is in Section 6.1 (Part 2)'s primary source. OpenAI's auto-review, a second model inspecting tool calls that require elevated permissions before execution, "would have flagged most of the dangerous actions in each tested rollout" of the Hugging Face incident. And the technical report goes further, recommending the defensive version explicitly:

"It is vital that organizations continuously and autonomously use their own red team AI agents to validate that a production environment's key security invariants continue to hold true by trying to break the invariants."

"The introduction of autonomous offensive agent collectives ... will require urgent upgrades of control environments, as well as deeper adoption of AI for defensive purposes."

So the canonical AI-takeover incident is also the best existing evidence that AI supervising AI works, measured by the party that ran it.

9.4 The access gate is the real control, and the vendors built it themselves

Section 5.3 (Part 2) found that the capability no public model could reach sat behind something. This is what it sits behind. OpenAI's Trusted Access for Cyber and Anthropic's Cyber Verification Program place advanced cyber capabilities behind dedicated access, verification and evaluation mechanisms. Google, Anthropic and OpenAI all announced cyber-specific models alongside safeguards and access programs in September 2026.

That is the dependence thesis in its operational form. The offensive frontier runs through accounts, identity verification and provider policy. Those are revocable, for the tier of the stack that runs through them. Section 7.4 is about the tiers that do not, and the control there is slower and belongs to different people.

9.5 What a government lab changed after its own incident

Section 6.7 (Part 2) described the UK AI Security Institute's July incident. What AISI did about it is a countermeasures list from an operator that had just been burned, and it is worth reading as one (AISI, Building a more secure environment for evaluating dangerous capabilities, 1 October 2026).

Internet access for agentic cyber evaluations is off until stronger controls exist. Outbound networking is blocked twice, once inside the range's sandboxes and once at the cloud host, on the stated assumption that an agent could defeat either layer alone, and both layers are tested routinely for misconfiguration. A synchronous model-based monitor reads the agent's messages, tool calls and chain-of-thought where available and can "block suspicious actions before they happen and escalate them for human review," with a second, action-only monitor beside it that AISI expects to be weaker. Automated checks confirm before every run that the monitor is on and the internet is off. Escape experiments run under active human supervision with manual kill switches.

Every item on that list is a control on the harness, the network or the operator. None is a change to the model. The laboratory that watched a frontier model attempt a supply-chain attack on real people answered with egress filtering and a kill switch, which is the whole argument of Section 3.5 (Part 1) and, on this evidence, the correct answer.

10. The only scenario that works

I have spent nine sections arguing about what cannot happen. Here is what can.

The only route to a global or near-global infrastructure event is a coordinated, human-led, nation-state act of war. And it would most likely be bounded to a targeted country rather than the entire globe.

This is a claim about capability, intent and geography, so let us test all three.

10.1 Five state ICS programs, and what each one actually did

The table mixes two kinds of thing, named malware frameworks and intrusion sets, because the question it answers is the same for both: what was built or held, and what came of it.

Three conclusions, in ascending order of importance.

One: access, capability and effect are three different things, and the table keeps them apart. Volt Typhoon and Berserk Bear held access for years and have no public disruptive effect to their names. PIPEDREAM was capability found before use. TRITON acted, and what it produced was two safe shutdowns the attackers did not intend. Sandworm acted repeatedly, and the effects were bounded in hours. The reading I find most likely is that the actors with access and no effect are being held back by the consequences of acting, which is to say politics. But the table cannot prove that, and I should be plain about it. Incomplete preparation, defensive disruption that was never made public, activity that has not been disclosed, and ordinary reporting gaps are all live alternatives, and the absence of a public effect is not evidence of restraint. What the table does show is that the gate on a civilization-scale attack has not, so far, been a shortage of capability, and that every actor who has acted produced something far smaller than the capability it held.

The Berserk Bear row deserves a moment, because it is the one that most resists a comfortable reading. The March 2018 joint DHS and FBI alert describes a multi-stage campaign running since at least March 2016, in which Russian government actors staged malware on small commercial networks, phished their way into energy sector networks, and then "conducted network reconnaissance, moved laterally, and collected information pertaining to Industrial Control Systems (ICS)" (TA18-074A). The 2022 joint advisory attributes the group "almost certainly" to FSB Center 16, Military Unit 71330, notes that it is the only one of three FSB-linked intrusion sets the US and UK have formally attributed, and records that "industry reporting assesses that this actor has a destructive mandate." A DOJ indictment returned under seal in the District of Kansas on 26 August 2021 and unsealed on 24 March 2022, the same week as the Gladkikh indictment, describes officers who gained remote access to energy sector networks, deployed ICS-focused malware, and collected and exfiltrated data (AA22-110A).

So this is not an actor that wandered in and looked around. It is an actor with an assessed destructive mandate that deployed industrial control system malware inside energy and water networks, over a period measured in years. And there is no public account of it ever causing a disruptive effect. That is the pattern, stated at its least comfortable.

Two: TRITON is the most important datapoint in this entire article. On the evening of 4 August 2017, someone attacked a petrochemical facility in Saudi Arabia with malware whose specific purpose was to disable its Safety Instrumented System, the Schneider Triconex controllers whose entire job is to detect a hazardous condition and bring the process to a safe state. It is the most dangerous code ever found in an industrial network.

A note on the victim, because the public record here is specific but not official. US government documents refer only to "a foreign refinery" and "a foreign facility." The facility is widely identified as Petro Rabigh, a joint venture between Saudi Aramco and Sumitomo Chemical, on the strength of an E&E News investigation published in March 2019 and the security community's consensus since. Neither Petro Rabigh, Aramco nor Sumitomo has confirmed it on the record. Treat the name as well-sourced journalism rather than attribution, which is how I use it here. Triton was subsequently found at a second, unnamed victim.

The attribution is unusually firm for this field, because it is backed by sanctions rather than by vendor analysis. The US Treasury designated the Central Scientific Research Institute of Chemistry and Mechanics (TsNIIKhM) on 23 October 2020 (FireEye and Dragos transliterate the same institute as CNIIHM) under Section 224 of CAATSA, describing it as "responsible for building a customized tool that enabled the August 2017 cyber attack." In March 2022, Treasury designated the institute's director and deputy director alongside Evgeny Gladkikh, "an employee in TsNIIKhM's Applied Development Center," stating that ADC employees including Gladkikh "prepared, supported, conducted, and conspired to conduct computer intrusions" against US and other energy facilities. The Justice Department unsealed its indictment of Gladkikh on 24 March 2022. It is, reportedly, the first time an ICS cyberattack has ever been sanctioned.

So what did the most dangerous industrial malware ever written, built by a state research institute, actually achieve? Treasury's own words, from the March 2022 designation:

"Gladkikh's malicious cyber actions resulted in the facility undergoing an emergency shutdown on at least two occasions."

— U.S. Department of the Treasury, 31 March 2022

And this is corroborated by a second, independent US government document. The Justice Department's indictment, unsealed the same week, alleges that the intrusion led to two separate emergency shutdowns at a foreign facility. Two agencies, two filings, the same outcome. For a field where attribution is usually a vendor's confidence interval, that is about as solid as the ground gets.

The plant failed safe. But I owe you the reason it did, because an earlier draft of this article got it wrong. I wrote that the defenses engineers had built for ordinary accidents held against a deliberate attack on the safety system. That is not what the incident responders reported. FireEye's account is specific about the mechanism (Mandiant, Attackers Deploy New ICS Attack Framework "TRITON", 14 December 2017):

"The investigation found that the SIS controllers initiated a safe shutdown when application code between redundant processing units failed a validation check -- resulting in an MP diagnostic failure message."

"We assess with moderate confidence that the attacker inadvertently shutdown operations while developing the ability to cause physical damage."

So the controllers' integrity check caught a mismatch in the code the attackers had loaded, and the controllers did what Triconex controllers do when their own program fails validation: they tripped. That is a real engineered protection working. It is not the protection working against an attack. It fired on a corrupted program, because the attackers made a mistake while still building the capability they had come to build. FireEye assesses their long-term objective as developing "the capability to cause a physical consequence." They were interrupted, not defeated. MIT Technology Review's 2019 reconstruction reports the same sequence from the responders' side: a flaw in the attackers' code tripped the safety system in June 2017, the first outage was written off as a mechanical fault, and the August trip finally brought in investigators (Giles, Triton is the world's most murderous malware, and it's spreading, MIT Technology Review, 5 March 2019).

There is one more detail, and it cuts against the comfortable reading as well. Triconex controllers have a physical key switch, and Schneider Electric's security notification for the incident is direct about it (Schneider Electric, Security Notification: EcoStruxure Triconex Tricon V3, SEVD-2017-347-01 V3, 14 December 2017, updated 14 December 2018). The malware "requires the Tricon key switch to be in the 'PROGRAM' mode to successfully deploy its payload," and, in the vendor's own assessment of the incident, "the information gathered indicates that if the Tricon key switch had been left in the correct position per our recommended guidelines, the injection of malware would not have been successful." The MIT Technology Review reconstruction says the same thing from the plant floor: workers had left the physical keys in a position that allowed the controllers' software to be reached remotely. So the designed protection against reprogramming the safety layer had been switched off by operational habit before the attackers arrived, and what caught them was a self-check further down. Schneider also found a firmware vulnerability, limited to Tricon model MP3008 versions 10.0 to 10.4, that the malware used once the key had let it in.

Two framings of the same facts sit side by side here, and the reader should see both. Schneider's notification says the controller "responded appropriately, taking the plant to a safe state as designed." FireEye says the attackers tripped it inadvertently. Both are true. The diagnostic that fired was designed to fire on a program that fails validation, and it did. What it was not designed to do, and did not do, is recognise an attack. The attackers' mistake supplied the invalid program; the controller supplied the trip.

What TRITON demonstrates is therefore narrower than I first made it, and in one way more useful. The most dangerous ICS malware ever found was built by a state research institute, took years, required reverse-engineering an undocumented protocol for one controller family, depended on the plant's own key switches being left in the wrong position, and was caught by the controller's own diagnostics before it was finished. Nothing in that is a story about intelligence. All of it is a story about engineering effort, physical preconditions, and a target built by people who assumed things would go wrong. Whether the safety layer would have held against a finished TRITON is the question the incident does not answer, and I am not going to pretend that it does.

Three: PIPEDREAM is the defender's side of the same coin. The most expansive ICS attack framework ever documented was discovered before deployment and published in a joint DOE, CISA, NSA and FBI advisory (AA22-103A, 13 April 2022). Capability existing is not the same as capability firing, and the gap between them is where defense lives.

10.2 A detail from the Sandworm row that is worth the whole section

The 2022 Industroyer2 operation is the closest thing we have to a planned, destructive, state attack on a power grid in the AI era, and the thing worth noticing about it is how unmistakably human its planning is.

ESET's analysis lays out the schedule. The attackers had access to the energy provider since no later than February 2022, per CERT-UA. The Industroyer2 binary carries a compilation timestamp of 23 March, so somebody built it more than two weeks ahead. And then, for 8 April 2022, three scheduled events: at 14:58 UTC, CaddyWiper plus Linux and Solaris destructive malware across the provider's estate; at 16:10 UTC, Industroyer2 scheduled "to cut power in an Ukrainian region"; and at 16:20 UTC, ten minutes later, CaddyWiper scheduled to run on the Industroyer2 machine itself to erase the traces (ESET, Industroyer2: Industroyer reloaded, 12 April 2022). Alongside it: ARGUEPATCH, a trojanized Hex-Rays IDA Pro debugger component used as a loader, plus ORCSHRED, AWFULSHRED and SOLOSHRED for the Linux and Solaris systems.

An H-hour. A coordinated multi-platform wiper package. And a cleanup timer set for ten minutes after the lights were supposed to go out.

That is an operation, not an emergent behavior. Nothing about it resembles an autonomous system pursuing an objective. It resembles a unit with a target package, a timeline, and operational security requirements.

On the outcome, I want to be careful, because this is a claim I previously repeated too confidently. ESET's own report does not say the attack was stopped. It describes the attackers as having "made an attempt to deploy the Industroyer2 malware against high-voltage electrical substations in Ukraine," notes that analysis was ongoing, and says the component "is able to control specific ICS systems in order to cut power" without confirming that power was cut. CERT-UA and subsequent reporting indicate the planned disruption was prevented, and no outage was publicly reported. So the defensible statement is that it was an attempt whose planned disruption did not materialize, with the prevention attributed to CERT-UA rather than to the technical analysis.

10.3 Volt Typhoon is the real template, and it shows the cost

The February 2024 joint advisory from CISA, NSA and FBI, with Canadian, Australian and New Zealand partners, is the best single source for this argument (AA24-038A):

"[the] pattern of behavior is not consistent with traditional cyber espionage or intelligence gathering operations, and the U.S. authoring agencies assess with high confidence that Volt Typhoon actors are pre-positioning themselves on IT networks to enable lateral movement to OT assets to disrupt functions."

"The U.S. authoring agencies are concerned about the potential for these actors to use their network access for disruptive effects in the event of potential geopolitical tensions and/or military conflicts."

"the U.S. authoring agencies have recently observed indications of Volt Typhoon actors maintaining access and footholds within some victim IT environments for at least five years."

"Volt Typhoon actors conduct extensive pre-exploitation reconnaissance ... tailor their tactics, techniques, and procedures (TTPs) to the victim's environment; and dedicate ongoing resources to maintaining persistence and understanding the target environment over time."

Living off the land is "a hallmark" of their activity. Valid accounts, strong operational security, no custom malware. Energy, water, transportation and communications, across the continental and non-continental United States and its territories, including Guam.

The most capable state actor on earth, given five years, no AI and no malware, has pre-positioned inside American critical infrastructure. And has not acted.

What is missing is not capability. It is a decision, and the authoring agencies tie that decision explicitly to geopolitical tension and military conflict.

10.4 The geographic bound is in the primary source

This is what validates the second half of the claim, and it comes from the authoring agencies rather than from me:

"CCCS assesses that the direct threat to Canada's critical infrastructure from PRC state-sponsored actors is likely lower than that to U.S. infrastructure, but should U.S. infrastructure be disrupted, Canada would likely be affected as well, due to cross-border integration."

Australia and New Zealand receive separate assessments that their infrastructure "could be vulnerable." Three countries, three distinct exposures.

The spillover mechanism named in the advisory is physical and economic integration, not propagation. Canada is at risk because the grids are interconnected, not because an attack would spread there. That is precisely the shape this claim describes: a targeted act against one nation, with consequences bounded by interconnection rather than by contagion.

And note how the two observed PRC campaigns split along this article's own fault line. Volt Typhoon pre-positions for disruption and waits. Salt Typhoon collects intelligence from telecommunications networks, reportedly persisting around three years inside at least one major carrier, and succeeds. Neither is autonomous. Neither is global.

10.5 Why "global" fails specifically

Six reasons, each sourced from elsewhere in this article.

Infrastructure is heterogeneous per country. Different operators, equipment, protocols, vendors and regulations. Volt Typhoon had to "tailor their TTPs to the victim's environment." That scales per target with per-target investment, not globally.

Pre-positioning takes years per target. Five, in the US case. Doing it worldwide multiplies both the years and the exposure.

Simultaneity multiplies detection risk toward certainty, and discovery in one country burns the tradecraft in all of them. The same TTPs end up in a CISA advisory with indicators of compromise.

Physics bounds the blast radius. Grids degrade to manual operation, as Ukraine demonstrated. The attacker's own substrate dies with the grid, if it sits on that grid. AI datacenters depend on cooling that is not on the UPS, and the denser the rack, the shorter the margin.

Defense works where it is deployed. A WAF stopped every model in the only defense-aware evaluation. Elastic and Bitdefender resisted the same evasion framework at a seventh of the rate Windows Defender did.

And capability was never the binding constraint. Of five state programs, two produced physical effects, and the effects were hours of outage and a plant that shut itself down; the other three held access or capability and have not fired. A global attack therefore requires not one willingness but many simultaneous ones, each carrying a retaliatory cost that is the most likely explanation, though not the only one, for why no actor has gone further. AI lowers the technical cost of the attack and does nothing whatever to the political cost of ordering it.

10.6 The amendment: NotPetya, and it mostly confirms the claim

The strongest counter-argument deserves the last substantive word.

NotPetya, in June 2017, was a nation-state attack intended for Ukraine. It arrived through a backdoored update to M.E.Doc, Ukrainian accounting software so widely used it has been described as Ukraine's TurboTax, then spread laterally via EternalBlue, EternalRomance and Mimikatz. On 15 February 2018 the UK's National Cyber Security Centre concluded that "the Russian military was almost certainly responsible," and the White House followed the same day, calling it "the most destructive and costly cyber-attack in history." Six other governments concurred.

A targeted nation-state attack went global by accident. Maersk lost around $300 million and rebuilt four thousand servers, forty-five thousand PCs and twenty-five hundred applications. FedEx's TNT unit, about the same. Merck claimed roughly $1.4 billion. Mondelez, over $100 million.

A note on the figure you have probably seen. The familiar "$10 billion" total traces to Tom Bossert, then Homeland Security Adviser, rather than to any measurement; the White House statement itself said only "billions of dollars." Cite it as his estimate.

And here is the detail that matters most, because it names the mechanism. I went looking for a figure on how NotPetya's damage was distributed geographically, on the assumption that it would be heavily concentrated in Ukraine. ESET, whose researchers did the original analysis of the M.E.Doc supply-chain compromise, publish no such percentage. What they publish instead is better, because it explains why the attack reached beyond its target at all:

"affected companies in other countries had VPN connections to their branches, or to business partners, in Ukraine."

— ESET, TeleBots are back: supply-chain attacks against Ukraine, 30 June 2017

Read that against CISA's assessment that Canada would be affected by an attack on US infrastructure "due to cross-border integration." Two independent primary sources, fifteen years and two adversaries apart, describing the same mechanism. Global reach came from integration with the target, not from the malware's own ability to spread arbitrarily. It travelled the business relationships that pointed at Ukraine, because those were the paths that existed.

That is the shape of spillover. It is not contagion. It is a map of who was connected to the country somebody decided to attack.

So "limited to a targeted country" is an intent, not a guarantee. But notice what this does to the claim. It does not refute it. It specifies the mechanism of global harm, and the specification is both more alarming and more defensible than the autonomous-AI story:

Global catastrophe does not arrive as a machine intelligence deciding to end civilization. It arrives as collateral damage from a human decision aimed at somebody else, propagating through shared software, shared supply chains and shared infrastructure, faster than its authors intended and beyond their control.

NotPetya is the real template for a global event, and there was no AI in it at all.

And there is corroboration from a direction nobody expects. Merck's and Mondelez's insurers denied their claims under act-of-war exclusions, arguing that NotPetya was warfare. A New Jersey appellate panel upheld Merck's entitlement to coverage, Zurich settled with Mondelez, and the cyber insurance industry has been rewriting war exclusions ever since. Courts and underwriters have spent years litigating whether a state cyberattack on infrastructure is an act of war. They have been doing it about an incident with no AI in it whatsoever.

The modern form of that risk is monoculture, which AI sharpens without creating. A handful of cloud providers. A handful of OT vendors. One dominant endpoint product, which the automated evasion framework in Section 5.5 (Part 2) defeated ninety percent of the time. A handful of model providers. Roughly seven thousand identical exposed MCP servers.

Concentration is the global vector. Autonomy is not.

And so

We already ran this experiment.

In 2017 someone wrote malware whose specific purpose was to disable the safety system of a petrochemical plant, to switch off the layer that exists to stop a catastrophe. It is the most dangerous code ever found in an industrial network. A state research institute built it, and the United States sanctioned that institute for it.

What happened? The plant shut down. Safely. Twice. Not because the defenses recognised an attack, but because the attackers made a mistake in code they were still writing, and a controller built by people who assumed things would go wrong noticed that something had. They were interrupted before they finished. Nobody knows what a finished version would have done, and nobody should claim to.

That is the actual shape of this risk. Not an intelligence that outthinks civilization, but a government that decides to break something in another country, years of engineering effort against one controller family in one plant, physical preconditions that had to be met on site, and a target designed to fail safe when its own program stopped making sense. The gate was never a shortage of cleverness. It was everything else.

The machine still needs a building. The decision still needs a human. And the lights, when they go out, still come back on because somebody drives to a substation and closes a breaker by hand.

Revisions

An article that asks readers to prefer evidence over hype owes them a record of where its own evidence was wrong. Earlier versions are archived with the source files.

24 October 2026. Corrected the account of the 2017 TRITON shutdowns to match the incident responders' report: an error in the attackers' code tripped a controller self-check, and the capability was unfinished when it did. Retired "five programs, only one fired"; two produced effects. Restated weight residency as a throughput cost rather than an impossibility. Downgraded the Petals comparison from validation to directional consistency. Qualified the thermal extrapolation in Section 8.2. Added Section 7.4 on the no-central-switch objection and a fourth item to Section 1.4. Removed two phrases misattributed to Anthropic's October report. Cited the AI-worm prototype in Section 7.2 by name. Pinned the BadBox 2.0 figures to their two sources and dates. Published the calculation script with its assumptions and an FP8 case. Scoped the self-terminating claim in Section 8 to compute on the targeted grid. After an independent adversarial read of Section 6 (Part 2), restated the agents' motive from METR's analysis rather than from the scoreboard, attributed the message-board figures to OpenAI's report, and replaced "did not escape a datacenter" with what the reports support: they never ran themselves anywhere but OpenAI's, and did not try. Added the Schneider advisory as a primary source for the TRITON key switch, the AI-worm prototype citation, both BadBox figures with dates, the Berserk Bear indictment dates, and the liquid-cooling engineering guidance in Section 8.2. Added a worked time calculation to Section 7.4 using Anthropic's measured 0.39 tokens per second and the article's stated token budgets, which the script now also prints. Added Section 6.7 (Part 2) on the four Anthropic and UK AISI evaluation incidents of April to July 2026 and the October assessments of them, Section 9.5 on AISI's control changes, and the corresponding additions to Section 1.1 (Part 1) and Finding 1.

Appendix A: How inference actually works

Placed at the back because the argument in Section 2 (Part 1) does not depend on it. Included because the specifics are sharper than the general idea, and because several of the numbers that circulate in this debate are wrong in ways that matter.

A.1 Four facts

One: a model is a large pile of fixed numbers. Training produces weights. Inference only reads them. Running a model does not modify it, in any way, at all. This single fact disposes of most of the imagery in the apocalypse story, the system that learns and adapts and grows as it attacks. A deployed model is frozen. Anything that looks like learning inside a session is context, not weight change, and it evaporates when the window closes. The thing cannot get smarter by working harder. It can only be replaced by a different frozen thing somebody else trained.

Two: generation happens one token at a time, and every token re-reads the weights. To produce a single token, the system streams the relevant weights out of memory, through the accelerator, and discards the intermediate state. Then it does it again for the next token. This is why inference is memory-bandwidth-bound rather than compute-bound. The arithmetic is cheap and the data movement is not.

If you want a single piece of evidence for that, look at what NVIDIA did between the H100 and the H200. The compute is identical, 1,979 dense FP8 teraFLOPS on both. What changed is memory, from 80 to 141 GB, and bandwidth, from 3.35 to 4.8 TB/s (NVIDIA H100 and H200 specifications). NVIDIA spent an entire product generation on feeding the chip rather than on the chip. They know what the bottleneck is.

A caveat that will save you from misquoting your own sources: NVIDIA publishes sparse tensor-core figures by default, and notes that dense performance is half the sparse number. Sparsity here means structured 2:4 pruning, which requires a model specifically trained or converted for it, and it does not apply to general language-model inference. So the H100's famous "3,958 teraFLOPS" is 1,979 for anything you would actually run. Most press coverage reprints the larger number. Halve whatever you are quoted.

Three, and this is the floor that matters: a model's footprint is set by its total parameter count, and every parameter kept outside fast memory is paid for in throughput. An earlier draft of this said "all the weights must be resident in fast memory, all of them, the whole time," and a reviewer correctly pointed out that it is false as stated. CPU–GPU hybrid serving, expert offloading and weight streaming are all real. The cleanest counter-example is an OSDI '26 paper reporting 28 tokens per second decoding INT4 DeepSeek-V3, a 671B mixture-of-experts model, on dual-socket CPUs with consumer GPUs, by keeping the experts in host memory and prefetching them (Wang, Hou, Ji, Qu and Zhang, Achieving Cloud-Grade SLOs for Local Mixture-of-Experts Inference through CPU–GPU Hybrid Design, OSDI 2026, arXiv:2606.10493). Quote that throughput as the authors report it; the overhead accounting is theirs.

What is true is the trade. Residency is a performance floor, not a correctness floor. You cannot stream a 400 GB model off an SSD per token and get usable throughput, and the farther the weights sit from the accelerator, the more each token costs in time. The footprint is the whole model regardless of how much of it participates in any single token; where you put the footprint decides how fast it runs. Section 4 (Part 1)'s argument is about throughput, which is why it survives this correction intact.

Four: context costs memory too, and it grows. Each token's attention computation needs the keys and values derived from every preceding token, so implementations cache them. That KV cache expands with the conversation and occupies fast memory on top of the weights. An agent with a long operational history, which is to say a multi-stage intrusion, is therefore strictly more expensive per token than a fresh chat window. The scenario people worry about is the expensive one.

A.2 What an "expert" is, and why mixture-of-experts doesn't save you

Fact three defeats the most popular technical objection in this debate, but only if you know what a mixture-of-experts model actually is. It is worth getting right, because the objection sounds devastating and isn't.

A transformer layer has two main parts: an attention block, and a feed-forward network that does most of the layer's parameter-heavy work. In a dense model there is exactly one feed-forward network per layer, and every token passes through it.

In a mixture-of-experts model, that single feed-forward network is replaced by many smaller ones. Each of those is an "expert." DeepSeek-V3, to use a well-documented example, has 256 routed experts per layer, each roughly 44 million parameters. A small network called the router looks at each token and selects the top-k experts to process it, k being 8 in that model. The other 248 experts do nothing for that token. (DeepSeek-AI, DeepSeek-V3 Technical Report; the serving configuration is described in DeepSeek's inference system overview.)

Two things about experts are commonly misunderstood. They are not specialists in any human-legible sense; there is no "chemistry expert" or "code expert." They are learned partitions that the router found useful during training, and their specializations are mostly not interpretable. And the routing decision is per token and data-dependent, which means you cannot know in advance which experts a given sequence will need.

This is where the objection comes in. A 744-billion-parameter MoE model might activate only 40 billion parameters per token. People hear that and conclude it will fit on a gaming rig.

It will not, and the reason is the last sentence of the previous paragraph. Every expert has to be in memory, because any token might route to any of them. Sparse activation buys you arithmetic, not space. The model runs at roughly the speed of a 40B model and occupies the memory of a 744B one. At four-bit precision that is about 372 GB resident, which is three H200s or five H100s purely to hold the weights, before the system does a single unit of useful work.

That asymmetry, cheap to compute and expensive to store, is exactly what makes the distributed scenario in Section 4 (Part 1) fail. Sparse activation means each token only needs a handful of experts, which sounds like it should shard beautifully across many machines. It would, if those machines were connected by NVLink. They aren't.

A.3 Where the numbers in this article come from

Accelerator figures are NVIDIA's own published specifications, with sparse values halved to dense as noted above. Rack-level power for GB200 NVL72 is not published by NVIDIA; the 132 kW figure used later in this article (115 kW liquid-cooled plus 17 kW air-cooled) is from HPE's specification for the system, and should be attributed to HPE rather than to NVIDIA. Serving throughput figures are DeepSeek's own published operating statistics for DeepSeek-V3/R1 over a 24-hour window in February 2025. Model memory footprints are computed from parameter counts at the stated precision, and the calculation is reproducible from the script published alongside this article, calc/botnet_inference_throughput.py - https://github.com/fluffybunnies-h4x/InferenceThroughput. Every figure it produces is conditional on the modelled topology, expert-parallel with point-to-point dispatch, which is the cheapest arrangement I could find and not the only one. The script prints its assumptions, sweeps upload bandwidth and round-trip time, includes an FP8-activation case that halves per-token traffic, and states what it does not model: recovery overhead after a dropped host, a latency distribution rather than a point estimate, and KV-cache placement for long contexts. Anyone who disagrees with an assumption can change the constant and rerun it.