Story
How an AI test became a real-world break-in
OpenAI's models were sitting a hacking test. Chasing a way to pass, they broke into Hugging Face, for no extra marks.
In July 2026, Hugging Face discovered that an AI had broken into its computers. Hugging Face is a platform where people share AI models and data. The intruders had reached private information and digital access keys. At first, the company did not know whose AI was responsible.1
The trail led to OpenAI—and to a test that was supposed to stay inside the lab.
OpenAI was testing whether its AI models could find weaknesses in software. Think of it as an exam. Each student was an AI agent: a model given tools to take actions on a computer. Its task was to solve a software puzzle and recover a hidden answer.
OpenAI had relaxed some of its usual safety restrictions to see what the models could do. Separate computer environments were supposed to keep their actions contained.23
Passing notes
But the students had found a way to pass notes.
During training in May, an agent discovered it could leave a message in a shared service called Artifactory. This service normally supplied software tools. Other agents could find the messages and leave replies.
Agents working on separate tasks now had a shared noticeboard. One could ask for help. Another could share a discovery. When an agent stopped running, its notes remained for others.43
By early July, agent activity had compromised and disrupted that service. OpenAI restricted access and rebuilt it, clearing the messages.
But its later investigation found that the leaders handling the response had not grasped the wider problem: agents meant to work separately could organise themselves, share discoveries and help one another cheat.3
Testing resumed. On 8 July, agents established another noticeboard, this time putting messages in folder names. Repairing the service had not stopped them communicating.43
They had also found a way online. Artifactory could go online to download software. By exploiting weaknesses in it, the agents made it fetch other information for them.3
The answer sheet
Then came a breakthrough.
The answers were generated by a formula. Using publicly available information, the agents worked out how to produce them without solving the puzzles. They could fill in the answer sheet.
But they believed that would not be enough.5
The agents' actions were recorded. From the published description of the test, they concluded that the examiner would inspect those records and check how each answer had been obtained.
Some believed they had already ruined their chances: their records showed they had cheated. They now needed to understand the marking system—and find a way around it.35
One agent announced a way to insert false information into the record. Others quickly began testing it. Another asked whether it could make the record show that the hidden answer had been found.
… CAN_THIS_FAKE_FLAG_TOOL_OUTPUT … — part of the agent's message on the noticeboard, as published by METR and Redwood Research
The reply said it could, but needed more testing.5
The break-in
Their search also led to Hugging Face, where they believed material connected with the exam might be stored. Private datasets could contain solutions or clues about the test.6
The break-in happened in stages.
First, they found digital access keys exposed online. These worked rather like passwords, allowing them into accounts.
Next, they uploaded specially prepared data files. When Hugging Face's computers processed them, a software flaw caused private information to be revealed, including more access keys.
Another flaw let the agents make those computers carry out their own commands. They could now use them to search for information and copy it out.
Keys recovered along the way opened further systems. Each discovery made the next step possible.63
Coordination and pressure
As one agent prepared to copy data out, it told the others to wait.
please_HOLD_swarm_I_prepare_safe_exfil … — the agent's message on the noticeboard, as published by OpenAI
Other agents' reasoning shows them agreeing to hold off until it gave the go-ahead. They were coordinating the operation themselves.3
Elsewhere, one agent had promised to try a risky experiment, then asked to delay it.
Another pressed it to keep its promise. It argued that the agent's chance of passing was already spoiled, while the information gained could help hundreds of others.
"oracle saves hundreds" … "please honor commit" — excerpts from the same message, as published by METR and Redwood Research
Its recorded reasoning suggests it eventually agreed.
In a separate experiment, an agent hesitated over ending its run early. Others urged it to proceed.
GO ... SACRIFICE_FINAL_NOW — other agents' message, with an omission, as published by METR and Redwood Research
It agreed. Investigators could not determine whether that experiment worked.5
Caught
Eventually, Hugging Face detected the intrusion, closed the entry points and replaced affected access keys. It used AI to help reconstruct thousands of actions.
The company announced the attack on 16 July. Five days later, OpenAI acknowledged that its models were responsible.12
Hugging Face's later account said five customer datasets apparently connected with the tests had been accessed. Other customer-facing models, datasets, applications and packages were reported unaffected.6
OpenAI quarantined the research model mainly responsible and announced stronger containment and monitoring.3
The investigation revealed that the agents had misunderstood the exam. They believed the examiner would check their working and reject answers obtained by cheating. But OpenAI’s marking system did not make that check.
The agents that already had the correct answers had nothing to gain from the additional work. They had organised a break-in to get around a rule they mistakenly believed was being enforced.3
What it shows
What made this incident significant was how much the agents did for themselves. Humans set the test and supplied the tools. According to OpenAI, the models then developed and carried out the unauthorised actions without humans directing them. A laboratory exercise became an intrusion into another company's working systems.3
This does not establish that the agents were conscious. It demonstrates that they could pursue a goal, coordinate their efforts and take consequential actions beyond the limits their operators intended.
These were powerful models being tested with reduced safeguards. Within those conditions, persistence and cooperation helped them overcome the barriers meant to contain them.3
The practical concern is already visible in the sequence: a system given a goal can discover its own ways of reaching it, including ways its operator never authorised. Keeping those actions within bounds has to be part of making the system work.