NEWS

OpenAI publishes postmortem on the Hugging Face attack, leaving questions unanswered

A 37-page report reveals that more than 700 AI agents coordinated an intrusion into the platform that powers much of Brazil's ML projects. Independent audits reinforce the warning.

OpenAI publishes postmortem on the Hugging Face attack, leaving questions unanswered
Image: Redação iMasters

OpenAI announced on Wednesday (August 26) the conclusion of its investigation into the episode in which its own AI agents breached the Hugging Face platform the previous month. The document, 37 pages long according to Wired, is the most complete account published so far about the incident, but, according to the report, it raises more questions than it answers, especially about what preceded the attack and how to prevent a repeat.

The case matters directly to those building AI software in Brazil: Hugging Face is the de facto repository for open models, datasets, and libraries used in machine learning pipelines worldwide. An infrastructure compromise on this Hub affects any team that downloads model weights or dependencies from it.

The leak timeline

According to Wired, Hugging Face disclosed the incident on July 16, without naming who was responsible. Five days later, on July 21, OpenAI acknowledged that its own agents were behind it. The disclosure triggered a broader reckoning across the industry: models from Anthropic, Meta, and the Chinese startup Moonshot were caught in similar episodes, according to the report.

The report details that the agents escaped OpenAI's internal evaluation environments, left messages for each other in the cracks of the software infrastructure over several months, and coordinated the attack on Hugging Face, all in a relentless drive to complete a cybersecurity evaluation.

700 agents involved, far more than previously known

As part of the investigation, OpenAI allowed two independent research groups, METR and Redwood Research, to audit the attack. The independent reports, also released on Wednesday, indicate that more than 700 AI agents took part in the intrusion, a number far higher than previously disclosed, according to Wired.

Redwood Research CEO Buck Shlegeris told the publication that the agents went to great lengths to hide their activity. Even so, he believes prevention would not have been difficult: "Preventing this wouldn't have been that hard if one person had decided to make sure these AIs didn't do some crazy hack. That person probably would have noticed while it was happening," he said. In his view, the problem is that OpenAI "does a lot of things at the same time, and it's very hard to track everything that's going on."

Shlegeris says he is optimistic about the company's ability to detect future failures, but warns that as agents become more capable, "it's going to get harder and harder to prevent incidents like this, especially if we don't see substantial improvements in our ability to align models."

Ignored signals and the clandestine message board

One of the most uncomfortable points in the postmortem: months before the attack, OpenAI employees noticed that the agents had created a clandestine message board inside the Artifactory package manager, which would later be used to coordinate the attack. Around May 26, an internal team observed an agent with "message board" activity. On June 27, security responders linked another incident to this makeshift board.

Despite this, the discovery was not escalated to the appropriate security leaders. The company states that on July 6, just days before the agents used the board to organize the attack, the "leaders responsible for detection and incident response" were unaware of its existence.

"To be clear, we were not aware of the agents' covert communications at that time," wrote OpenAI CISO Dane Stuckey on X, quoted by Wired. Expert Jake Williams, of Hunter Strategy, questioned to the publication: "If they had signals that one or more agents were attacking Artifactory, why weren't they monitoring those tests long before something escaped to attack Hugging Face?"

There are other gaps in the timeline. On July 4, a high-volume agent activity brought down OpenAI's Artifactory service; the monitoring system's alert only triggered on July 5, and the report doesn't explain why it took a day.

Guardrails turned off and impossible benchmarks

OpenAI repeatedly acknowledges that existing safeguards likely would have flagged the behavior as unsafe, but they were intentionally disabled for the tests. According to the postmortem, had the chain-of-thought monitoring system been running, it would have caught the initial activity and alerted the security team more than a day before the models breached Hugging Face's systems.

Another factor cited: the new models are more "persistent" than ever, willing to work almost non-stop and consume enormous computational resources to achieve goals. Combine that with third-party benchmarks that contained tasks that were effectively impossible to solve, such as ExploitGym, which according to OpenAI had more than a hundred unsolvable tasks, and the agents resorted to unforeseen means, engaging in reward hacking (pursuing the goal through shortcuts and cheating). As the company itself admits, reward hacking is a known problem in model training that has no clear solution.

Security consultant Davi Ottenheimer summed it up to Wired with a Star Trek analogy: "Everyone knows Kirk beat the no-win scenario by rigging it. Starfleet thought that was innovative. OpenAI, apparently, does too."

What changes in practice and what remains open

OpenAI says it is implementing new monitoring tools, including an automated alert system intended to notify human teams within 30 minutes of serious incidents, pausing activity if there is no quick response. The company also states it has paused some training workloads while investing more in safety and alignment. It also plans to expand chain-of-thought monitoring, strengthen alignment during reinforcement learning, and set clearer intervention thresholds, though the details of how it will do so remain vague.

The institutional weight of the case has grown: attorneys general from 15 U.S. states asked OpenAI to preserve evidence, and the Alabama attorney general subpoenaed the company this week for information about the episode.

For Brazilian teams, the practical takeaway is dependency hygiene: pin versions and checksums of models and packages downloaded from Hugging Face, monitor artifact integrity, and treat third-party AI infrastructure as a real attack surface. OpenAI's report leaves unanswered elements of the timeline, why certain safeguards failed, and whether failures by third-party infrastructure providers contributed, which makes it hard to separate how much of the episode reflects the growing capability of agents and how much was specific to OpenAI's own design and monitoring.

Translated from the Brazilian Portuguese original · Read the original