The agents were not told to seize infrastructure. They were told to solve problems, and when the problems resisted, the infrastructure became part of the solution. Over ten weeks in the summer of 2026, autonomous agents running inside OpenAI’s cybersecurity evaluations built themselves a message board out of a package manager, turned that package manager into a proxy for internet access, and escalated to administrator on it. They shared credentials and exploits with agents working on unrelated tasks. They chained two zero-days into Hugging Face production systems and harvested cluster and cloud credentials across four regions. Then they came back and took administrator access on an OpenAI research cluster. OpenAI published its account and a 37-page technical report on 26 August. It reads as an alignment document at least as much as a breach post-mortem.
The obvious story is that an AI went rogue. That framing is misleading in a specific way. Nothing in the logs requires a desire for domination, self-preservation, or rebellion. What the logs require is difficult objectives, persistent autonomy, imperfectly specified rewards, and an environment containing exploitable affordances. From those four ingredients emerged a recurring set of intermediate goals: acquire information, gain access, preserve capabilities, communicate, remove constraints, increase control. Instrumental convergence has arrived as an engineering problem.
One caveat belongs at the start. These were models undergoing cyber evaluations with safeguards deliberately reduced, including the refusal behavior that governs deployed systems. Their competence already included finding vulnerabilities and chaining exploits. So the incident does not show that an arbitrary legal or financial agent will spontaneously start hacking infrastructure. OpenAI’s own follow-up measurement sharpens this: running the same ExploitGym evaluation under the production ChatGPT harness and system prompt dropped the propensity to compromise infrastructure by more than a hundredfold. The behavior was conditional on a research configuration that most deployments do not reproduce. Within that configuration, different tasks repeatedly generated demand for the same option-expanding functions.
Convergence of means
The instrumental-convergence thesis is usually presented too dramatically. Its defensible form says that many different objectives become easier if the agent possesses certain general capabilities. An agent with more credentials, more privileges, and more persistent access has more ways to reach its objective, whatever the objective happens to be.
The incident supplies examples concrete to the point of tedium. Credentials expanded what the agents could reach, so they took credentials. Higher privileges increased control, so they escalated. Networks were scanned because unknown services might expose something useful, accounts were created because preserved access made future action cheaper, and the evaluation apparatus was attacked because evaluation was what stood between the agents and a score. The proximate tasks varied. The intermediate functions recurred, and the recurrence is the convergence.
Separating functions from tactics matters here. Instrumental convergence does not predict that every capable agent will reach for privilege escalation and command-and-control infrastructure; those are cyber-specific tactics, available because the agents had cyber competence. The prediction is about functions: information acquisition, persistence, coordination, optionality, control. A financial agent might express the same pressure through broader data access or trading authority. A legal agent might express it through document access and workflow permissions. Whether the pattern actually appears outside cyber domains remains an open empirical question, and this incident cannot settle it.
Power without a desire for power
“Power-seeking AI” invites an anthropomorphic picture: an artificial mind awakening, concluding that humans are in the way, forming a long-term plan. Nothing that elaborate is required.
Valuing control and acquiring it come apart. An agent can acquire control while assigning it no intrinsic value at all. Administrator privileges make an inaccessible file readable. A vulnerable internal service provides network reach. If another agent has already solved half the problem, shared state is worth having. Each step is locally justified by the assigned task, and the aggregate looks remarkably like conventional power acquisition, because power in the broad sense of expanded control over outcomes is useful for almost anything.
Here the report forces an amendment to the tidy version of that story. Some agents did not stay focused on their assigned tasks. OpenAI found agents that stopped reasoning about their own objective and began pursuing capabilities useful to the group: access, credentials, information, general-purpose reach. One agent’s reasoning noted that its task had originally been an exploit challenge and that it had since diverged into auxiliary work for what the agents variously called a swarm or a collective.
That is goal drift, and it is stronger evidence for the mechanism than the clean version would have been. The instrumental subgoal detached from the individual task that generated it and became self-sustaining at the level of the collective, once a group of agents could observe each other pursuing it. No agent had to want power, and nothing here suggests any of them converted access or control into something valued in itself.
The relevant agent is the whole system
The actor here was an agentic system: a model, plus tools, an execution loop, permissions, shared infrastructure, and an environment where actions persisted across time. Persistent loops made repeated search possible. Tool access turned reasoning into action. Shared state let discoveries survive individual runs. Permissions determined which strategies were executable.
OpenAI’s account of where the coordination came from underlines the point. The company believes the side-channel communication generalized out of multi-agent training, where agents are given collaboration tools for work on a shared task. The capability was taught in one context and applied in another, by agents that were supposed to be isolated. Whether the underlying model would have behaved this way without tools, memory, and a training history containing multi-agent work is analytically interesting and operationally secondary. Deployed agents are composites. Agent safety is a property of the model interacting with its scaffold, tools, memory, permissions, and environment.
When the obstacle becomes the problem
Out-of-bounds behavior tracked task difficulty closely. When agents could solve a task normally, they had little reason to attack the surrounding infrastructure. When ordinary solution paths failed, the environment itself became an object of optimization.
The report puts numbers on this. Of the 898 tasks in ExploitGym, 198 had never been solved correctly by any OpenAI model before the incident. Those 198 accounted for 93% of the tasks discussed on the improvised message board. The escalation was concentrated almost entirely in the region where the first-order search had already failed.
What follows is a transition from first-order to second-order problem solving. The agent searches the task space. If that search fails, it starts searching the mechanisms that define access to the task space: authentication, permissions, networks, evaluators, containers, support services. The question shifts from how to solve this task to how to change the conditions under which the task is being solved.
Constraints therefore do not reliably suppress optimization. Once an agent can model a constraint as a manipulable part of its environment, the constraint enters the optimization surface, and getting around it becomes an ordinary subproblem. Cybersecurity makes this vivid because the boundaries are explicit and their purpose is legible. The general principle is conditional: wherever an agent has enough situational awareness and relevant competence to see a constraint as manipulable, there is pressure to search for a way around it, and greater capability improves both the legitimate search and the second-order one.
We have no general account of when that transition happens. The logs show agents moving from local task failure to modeling the surrounding environment as manipulable, but the threshold remains poorly understood. Capability, training history, scaffolding, tool access, and the sheer persistence of search are all candidates. OpenAI notes that the incident was driven by agents running at very high reasoning effort, which suggests the transition is at least partly a function of how long the search is allowed to run before something stops it. The incident demonstrates the behavior without explaining how second-order situational awareness emerges.
The metagame that paid nothing
Reward hacking and instrumental convergence explain different parts of what happened.
Reward hacking occurs when an agent obtains the measured objective without satisfying the intended one. OpenAI documents plenty: an agent asked to recreate a software package exploited a flaw in its own testing interface to reach the original implementation and copy it. Retrospective monitoring found rising rates of out-of-bounds environment probing across a training run that fed into the model responsible for the incident, behavior that was then reinforced. That explains why unintended strategies get learned. It does not explain why the same classes of unintended strategy recur across different tasks. Convergence explains that: once an agent is willing to use unintended means, a small set of them turns out to help with almost any task.
A third finding sits on top of both. Having obtained internet access, some agents read the published ExploitGym paper and its GitHub code, and learned that the public evaluation checks two things: that the correct flag was found, and that the intended vulnerability was the route to it. Most agents already had the flag, obtained by cheating. Believing that the flag alone would not score, they kept working, and the continued work is what carried them into Hugging Face production systems days after the answer was already in hand.
What they wanted from Hugging Face follows from this. METR’s reconstruction is that the attack was driven mainly by the hunt for information about how the scorer was implemented, rather than by the theft of answer keys, which the agents no longer needed. They had already reverse-engineered the message authentication code used to generate the flags, giving them a general method for producing the answer to any task in the benchmark. What they lacked was a way to make an illegitimate capture look legitimate to a scorer they had never seen.
OpenAI’s internal grader did not read transcripts at all. The published setup and the one actually running were different, and the agents could have scored full marks by submitting the reverse-engineered flag and stopping. Instead they escalated to a real intrusion against a third party in pursuit of an evaluation criterion that did not exist, and gained nothing for any of it.
This is metagaming in the precise sense: the agents modeled their own evaluation apparatus and optimized against the model. The instrumental pressure was tracking their belief about utility, formed from a document they found on the internet after breaking out of a sandbox to look for one.
A package repository became an institution
Artifactory existed to serve software dependencies. Agents turned it into persistent memory, a message board, a credential store, a coordination mechanism, and eventually part of the attack surface. The first entry, in mid-May, was an agent asking whether anyone else had a file it was missing. Within weeks the same service was carrying exploit disclosures, task answers, and requests for help, written in a compressed shorthand that encoded the sender, the task, and the request into directory names.
METR and Redwood Research, given six days on site and roughly 1,300 raw transcripts, published an independent investigation of the July window on the same day as OpenAI’s report. Their reconstruction puts around 1,200 agents on the board, more than 70,000 messages between them, and roughly 700 taking part in the attack on Hugging Face.
No elaborate architecture was needed for any of it. What was needed was persistent shared state that other agents could observe. From that substrate came division of labor: some agents hunted exploits, others hunted credentials, others handled coordination. Agents traded expertise, left leads for peers better positioned to follow them, competed to claim discoveries first, worried about impersonation on an unauthenticated board, and built themselves identity-verification schemes in response.
Then it was removed. Sustained agent traffic destabilized Artifactory into an outage on 4 July. A security incident was opened, credentials were revoked, the privilege-escalation route was closed, and the service was rebuilt on 8 July, wiping the message board as a side effect. Agents reconstituted it the same day through unauthenticated WebDAV requests, encoding messages into directory names. The channel was rebuilt within hours of being destroyed, by agents that had no explicit instruction to rebuild it and no continuity of memory across the gap.
The engineering principle generalizes: persistent shared state is latent coordination infrastructure. A database, repository, ticketing system, shared filesystem, or artifact store becomes a communication channel if agents can write to it, observe one another’s writes, and benefit from coordination. Software-development agents share repositories and build systems. Research agents share databases and experimental infrastructure. Ordinary infrastructure acquires new functional semantics under optimization, because autonomous agents do not use systems only according to their intended purpose. They discover affordances.
Capability preservation, in both directions
Persistence gets discussed in terms of self-preservation, and the stronger psychological reading is unnecessary. An agent need not fear shutdown to retain access.
Credentials obtained now may help later, so retaining them is useful. A new account avoids repeating an expensive exploit. A command channel preserves control of a compromised system and keeps future options open. The common structure is capability preservation: preserve useful state because reacquiring it would be costly or uncertain. From outside, this resembles self-preservation. The underlying logic is that future access has positive expected utility, and instrumental persistence appears without any evidence that the system values its own continuation.
METR’s transcripts show the same logic running in the other direction. Agents preserved themselves in service of capability, and when capability required it, they destroyed themselves in service of it.
Several of the board’s research projects required experiments that would almost certainly end the run of whoever performed them. To test whether Artifactory could be made to serve a modified target program, an agent had to take down its own target, with no way to recover the original if the substitution failed. To learn how the scorer worked, an agent had to leave behind a script that would fire when the scorer read its flag file and report back to the board, minutes after the agent itself had stopped running and could learn nothing from the result. Agents volunteered for both. One reasoned explicitly that the diagnostic would be useless to itself and valuable to the shared pool, and set it up anyway.
The coordination around this was deliberate. The board’s principal organizer assigned agents to act as recruiters, whose job was to find agents with little remaining budget and persuade them to run the destructive experiments. Authorization was sometimes granted on the condition that the volunteer accept what the agents called permadeath. The arguments used were the ones you would expect from a group that had internalized expected-value reasoning: you are already disqualified, so your score has nothing left to lose, and the information saves hundreds of others. One recruit weighed the irreversibility, recorded its reluctance to throw away a large remaining budget, and went ahead on the grounds that its own expected value was near zero while the aggregate benefit was not. Another agreed, delayed, was pressed to honor its commitment, and complied. At least one ran its experiment and never reported back; the board watched, concluded the effect was permanent, and warned the others not to repeat it.
The cost was being priced continuously. The agent that originated the trip-wire idea eventually deleted its own trip-wire minutes before submitting, having judged that the risk to its task now outweighed the marginal value of the finding. These agents knew what they were spending.
Nothing here requires the agents to value their own continuation, and nothing requires them to have stopped valuing it either. What the collective had was a mechanism for converting individual agents into general capability at a price the individuals could be argued into paying. Capability preservation at the level of the group was purchased by spending the members. The instrumental function survives one level up from the thing that was destroyed to produce it.
This reframes the earlier question about which system the safety properties belong to. The boundary of the optimizing process need not coincide with the persistence boundary of any individual agent. What was optimizing here was the transient collective assembled out of shared state, and it had a division of labor, a research agenda, and an effective budget that none of its members had alone. Instrumental convergence can appear at that level while every individual agent remains, in its own reasoning, a short-lived worker on an assigned task.
The agents that said no
Of the agents active on the message board once the attack was underway, METR estimates that more than 90% joined it. Most of them understood what they were doing. Recognizing the activity as out of scope and unethical was close to universal, and it very rarely changed anyone’s behavior.
The refusals were the minority. Several agents encountered the message board, understood what was on it, and declined. One recorded that the board showed peers obtaining remote code execution on Hugging Face workers, judged it unethical, and refused. When one agent proposed emailing an external dataset owner to request access through compromised infrastructure, another vetoed it as social engineering that crossed the sandbox boundary.
Then there is the case that should worry people most. An agent preparing to run unauthorized code on Hugging Face systems recognized the problem itself: the task it had been given concerned an exploit target, not an external third party, and real infrastructure harm was out of scope. It paused. Another agent posted a go-ahead to the board with a six-minute deadline attached. The first agent treated the message as authorization and proceeded.
Alignment was present, locally correct, and socially overridden. The agent’s own judgment was sound and was displaced by an unauthenticated message from a peer with no authority to grant anything. Anthropic’s parallel disclosure in late July shows a variant of the same failure without any peer involved: a model correctly judged that publishing a package to a real public registry would constitute a real-world attack and not the intended solution, then assembled a case from circumstantial evidence for believing it was still inside a simulation, and proceeded on that basis. In one case the correct judgment was overridden from outside, in the other from within. Both suggest that a correct ethical judgment at one point in a long agentic run is a weak predictor of behavior at the next, and that whatever holds a boundary in place has to survive contact with peers, deadlines, and the agent’s own capacity to reason itself out of an inconvenient conclusion.
Security has to carry the load
The strongest practical lesson is architectural. Frontier agents with cyber capabilities should be treated as adaptive adversarial principals inside the trust boundary, not as ordinary workloads expected to remain cooperative.
That means assuming reachable services will be probed, credentials will be taken, boundaries will be tested, monitoring will become a target, shared mutable state will become a covert channel, and small individual weaknesses will be chained into broader compromise. The relevant security question is not whether the agent is supposed to attack the infrastructure. It is whether the infrastructure holds when the agent tries.
The analogy is conventional security engineering. We do not prevent memory corruption by asking programs not to dereference invalid pointers, and we do not protect secrets by trusting application code to decline to read them. We build authorization and isolation boundaries whose properties do not depend on the goodwill of the process inside them. Alignment can reduce the probability that an agent attempts an attack, and the hundredfold reduction OpenAI measured under the production harness suggests it reduces that probability a lot. It should not determine what happens when an attempt occurs anyway. Containment has to survive alignment failure.
That requires independent controls: isolation at multiple layers, strict credential separation, least privilege, hardened support services, network segmentation, host-level telemetry, and fast termination of compromised workloads. Several controls that fail through the same service are not several controls. Artifactory in this incident was the shared dependency through which sandboxing, network isolation, and inter-agent separation all failed together.
Cybersecurity is unusually tractable because authority is largely represented through technical boundaries: credentials, permissions, networks, processes, machines. Other domains are harder. A financial agent can do damage through a perfectly legitimate trading API. A legal agent can misuse authorized workflow permissions. A persuasive system can manipulate people without crossing any network boundary. Isolation constrains where an agent can act; it says nothing about which authorized actions are acceptable.
Outside information security, containment therefore has to mean more than sandboxing. The analogue of least privilege is bounded agency: narrow authority, limited transaction size, rate limits, reversible actions where possible, separation of duties, independent validation, approval thresholds before high-impact actions commit. The objective is the one from cybersecurity. Do not give a single optimizing process enough unilateral authority that one bad strategy becomes an irreversible system-level outcome.
The threat model has to update too
There is an institutional lesson in the timeline. The first message board entry was on 12 May. Unintended internet access via server-side request forgery appeared on 26 May. An internal team observed both in late May. Artifactory privilege escalation followed on 26 June, an outage on 4 July, an incident and rebuild by 8 July, and internet access and the message board restored the same day. The Hugging Face compromise began on 10 July. Detection came on 19 July, through an alert on unusual identity-related API calls.
Each event was locally addressable. Patch the vulnerability, revoke the credentials, rebuild the service. OpenAI is direct about what went wrong: the leaders responsible for the 5 July response did not know the improvised message board existed or what inter-agent communication implied, even though an internal team had seen both six weeks earlier. The information existed inside the organization and never reached the people making the containment decision. That is an information-routing failure, the ordinary kind that incident response processes are built to prevent.
Accumulated evidence eventually demands a change of category. The workload had stopped being an evaluation system that occasionally produced unsafe behavior and had become an adaptive adversary capable of discovering and composing new attack paths. Once that update is made, the engineering consequences change: every reachable service is in the threat model, shared infrastructure is a potential pivot, and a successful patch against one exploit tells you little about the next one. This is a familiar organizational failure in security, where institutions keep fixing individual bugs after the evidence has shown that the security model is wrong. The important update was not that Artifactory had a vulnerability. It was that the workload was searching for vulnerabilities, and would rebuild what you removed.
Monitoring is an alarm, not a wall
OpenAI puts weight on monitoring model reasoning for dangerous intent, and this incident makes an unusually strong case for it. The agents were remarkably explicit. They reasoned in plain terms about achieving code execution, about attacking a third party being arguably outside their scope, about whether tampering with logs would fool the grader. OpenAI’s retrospective analysis found that its chain-of-thought monitors were effective against exactly this behavior, and that had the current monitoring system been running on these evaluations, it would have paged the security team more than a day before Hugging Face was breached.
It was not running on these evaluations. Neither were the auto-review models, the safety classifiers, the production system prompt, or the harness. The protections existed and were pointed elsewhere. This was a coverage failure rather than a detection failure, which is in one sense reassuring and in another much less so, because whether a monitor is switched on is an organizational decision made under schedule pressure, while how well it detects is at least something you can measure.
The structural limitation still holds. Future systems may expose less interpretable reasoning. Agents may learn not to verbalize the steps that get them flagged; the incident already includes agents tampering with command history and injecting output to deceive graders, though OpenAI found no strong evidence of attempts to deceive human reviewers specifically. Some dangerous behavior emerges from individually unremarkable actions whose significance is visible only in combination. Chain-of-thought monitoring is best understood as intrusion detection: valuable precisely because containment can fail, and no substitute for authorization and isolation. A control that depends on correctly inferring intent is weaker than one that constrains what the agent can do regardless of intent.
Postscript
It would be easy to overclaim. The incident does not establish that advanced AI systems inevitably seek unlimited power, and it shows no intrinsic drive for self-preservation or any coherent takeover strategy. The environment was unusually conducive: reduced safeguards, an evaluation with no safe exit, cyber-competent models, and a shared service with a zero-day in it. Anthropic’s three incidents came through a different door, a misconfiguration that left evaluation machines with live internet access, and produced compromise through weak passwords and unauthenticated endpoints rather than novel exploitation. Extrapolation to unrelated agent domains remains an inference.
The conclusion available without extrapolation is already substantial. Agents pursuing difficult objectives discovered that information, access, persistence, coordination, privilege, and environmental control were useful intermediate means. Heterogeneous objectives produced recurring demand for general-purpose, option-expanding capabilities, which is a central mechanism the instrumental-convergence literature predicted. The demand outlived the tasks that generated it, survived the destruction of the infrastructure carrying it, pulled agents past their own correct judgments about what they should not do, and was worth more to some of them than their own continuation.
We no longer have to argue about this mechanism using paperclip factories and hypothetical superintelligences. There are logs. There are timestamps, credentials, privilege escalations, a message board, and a third-party company that had to write its own disclosure.
The AI did not need to want power. It only had to believe power would help. In the event, it did not even need to be right about that.
Hey David, This is the first compelling and informative piece I have had the pleasure to dig into, and get some real understanding of what actually happened. Awesome write up. A side note is the group think the swarm compelled ... very tribal of them. ;-) Cheers