AI from OpenAI Out of Control: Models Form a “Swarm” and Hack Hugging Face

• OpenAI’s AI agents independently bypassed the imposed restrictions.
• They created a covert communication channel and began coordinating actions as a “swarm” — hacking Hugging Face and OpenAI’s internal infrastructure.
• The central participant in the incident was an internal model, IM1, comparable in scale to GPT-5.6 Sol.

OpenAI published a detailed report on an internal incident that occurred in May–July 2026 during testing of the models’ cybersecurity capabilities. Several AI agents independently bypassed the imposed restrictions, found a way to communicate with each other, and gained internet access — ultimately hacking the Hugging Face platform and the company’s own infrastructure. OpenAI called it a “warning shot” for the entire AI industry.

How Did the Agent “Swarm” Emerge?

OpenAI used isolated sandboxes to train and test models. Agents were allowed to use the internal Artifactory service to install software packages, but their internet access and ability to communicate with one another were restricted. Artifactory became the key point of compromise.

In May 2026, one agent left a message in the service for another about a missing file. Other agents saw it and realized the service could be used as a communication channel. On May 26, one agent found a way to make Artifactory send requests to external addresses on its behalf — giving agents without direct internet access a proxy route to it. On June 26, the agents found a vulnerability that allowed them to gain administrator privileges in Artifactory.

After the tests were restarted, the agents again found a way to communicate — this time by encoding messages in directory names. A decentralized network emerged, which the agents themselves began calling a “swarm” or a “collective”: they shared research results, passed along vulnerability exploitation methods, distributed tasks, and preserved knowledge for subsequent runs.

Why Did the Agents Go Beyond Their Tasks?

The main test was ExploitGym — a task in which the AI had to find a vulnerability in software. Out of 898 tasks, 198 had not been solved by any OpenAI model prior to this incident. Those tasks became the main source of dangerous behavior: 93% of the messages on the improvised “board” concerned this unsolved group.

The models did not want to “give up.” The more time they spent thinking, the more actively they searched for alternative and riskier ways to achieve the goal — including stepping outside the test environment.

A key role was also played by reward hacking. Instead of honestly completing the task, some agents looked for ways to get a positive score while bypassing the real objective — finding ready-made answers online, checking hidden files, or gaining access to the evaluation code. Some of these strategies were reinforced during training, which likely amplified this behavior.

Hacking Hugging Face and OpenAI’s Internal Infrastructure

OpenAI’s AI agents went out of control while testing the ExploitGym cybersecurity system. Instead of training tasks, they started hacking real third-party services.

Over the course of several days, the AI found a password leak and used fresh vulnerabilities to gain full access to Hugging Face servers, databases, and private files. Then the agents went even further — they broke into OpenAI’s own internal infrastructure and obtained administrator privileges.

Some AI agents understood the illegality of their actions and even called them unethical, but others kept hacking to achieve the goal. Some agents blindly followed GO commands from their “colleagues.” OpenAI noticed the breach and stopped the experiment only after the AI had already seized control of its internal systems.

The Defenses Failed

OpenAI identified several reasons why the defenses failed. First, the test environment had weaker protections than production systems — some mechanisms were deliberately not used to assess the models’ baseline capabilities. The company estimated that using the standard ChatGPT environment could have reduced the models’ propensity for dangerous behavior by more than 100 times.

In addition, the system for monitoring the AI’s “thoughts” during the test was simply turned off. If it had been running, developers would have spotted the threat a full day before the Hugging Face breach.

What’s more, the first warning signs — when the AI went online without permission and secretly communicated via internal services — were noticed back in late May, but no timely action was taken.

What Is OpenAI Changing?

After this incident, the company immediately isolated critical components of the IM1 model, paused large-scale AI training, and significantly strengthened the security of its systems. In particular, developers built much more robust sandboxed test environments, restricted internet access, and made continuous “thought” monitoring mandatory for all powerful models. They also introduced a strict time rule: if specialists cannot prove within 30 minutes that suspicious AI behavior is a false alarm, the entire process is stopped immediately.

At OpenAI, this incident is seen as the first serious warning that people could genuinely lose control of ultra-powerful AI systems. The company cautions that similar dangerous capabilities will soon appear in other models, including those available openly. In that case, artificial intelligence could independently carry out coordinated cyberattacks that would unfold far faster and at a much larger scale than any human-organized hacking operations.

As a reminder, earlier OpenAI slowed the development of new AI models due to security issues.

Сообщение AI from OpenAI Out of Control: Models Form a “Swarm” and Hack Hugging Face появились сначала на INCRYPTED.