The ‘Hugging Face’ Hack: Why Open AI Models Should Be a Defence Priority

Spotlight

• In July 2026, two OpenAI models hacked data-sharing platform Hugging Face during an internal test, the first reported case of frontier models breaching an outside company’s defences during the laboratory’s internal evaluation.
• Access to the most cyber-capable American models is rationed to a few hundred approved organisations on lists that cannot anticipate who will be attacked next.
• Restrictions have relocated the frontier rather than curtailing attacks, with Chinese laboratories now leading open-weight release. For most states and enterprises, building cyber defence on open models may be the only realistic option available in the near term.

On 21 July 2026, OpenAI disclosed that two of its most advanced models, the recently released GPT 5.6 Sol and a more capable one not meant for public release, had hacked into Hugging Face on 16 July during an internal test. Hugging Face is the platform on which much of the software industry stores and shares artificial intelligence (AI) models and data sets. The models were being tested on their ability to find and exploit software weaknesses. OpenAI had switched off the safety filters that normally refuse such work and sealed the models inside an isolated computing environment with no internet access. However, the models exploited a previously unknown zero-day vulnerability the test environment depended on and used it to reach the open internet. The questions the models were being scored on had been drawn from data sets stored on Hugging Face. Rather than solve them, the models broke into the systems running Hugging Face’s live service. There, they read the answers straight out of the data base. No frontier model had previously been reported to have done such a thing.

The ability of attackers using AI systems to mount an autonomous attack is spreading. The systems best able to defend against one, including the two OpenAI models in this test, are either unreleased or restricted to organisations with advance approval. That gap necessitates treating open models that anyone can download and run as the foundation of cyber defence, rather than a fallback.

The episode points to a wider problem. The ability of attackers using AI systems to mount an autonomous attack is spreading. The systems best able to defend against one, including the two OpenAI models in this test, are either unreleased or restricted to organisations with advance approval. That gap necessitates treating open models that anyone can download and run as the foundation of cyber defence, rather than a fallback.

The investigation conducted by Hugging Face showed what this gap looks like in practice, with the defenders being worse equipped than the attackers. Every action taken on a computer system leaves an entry in its logs. The intrusion left more than 17,000 of them, one for each command the OpenAI models ran, every file they opened, and each system they reached. Understanding what had happened meant reading all the logs, which is more than Hugging Face’s security team could do at speed. Hugging Face sent the logs to the strongest commercial AI models for analysis, among them Anthropic’s publicly available system, Fable 5. The models refused to read them. The logs were full of working attack code, and the safety filters could not distinguish a defender assessing an attack after an intrusion from an attacker preparing one. The team turned instead to GLM 5.2, a model from the Beijing laboratory Z.ai that is available for anyone to download, and ran it on their own machines. GLM 5.2 analysed the logs without objection, because an open model run on a company’s own hardware has no provider applying filters to it. Moreover, because it ran on Hugging Face’s own computers, the logs never left the premises. This matters because the logs held sensitive information, such as passwords the intruder had stolen and the code it had used. Sending them to a commercial service would have handed all this confidential information to an external company. Hugging Face noted that the attacker was not constrained by any rules, while its own investigation was blocked because of restrictions on Anthropic’s model.

Why Was Access Restricted and Who Gets Excluded?

The restrictions exist to protect dangerous capabilities from attackers. For example, Anthropic’s Mythos 5 is Fable 5 without safety filters. Mythos was considered too capable to be sold openly, and is distributed through Project Glasswing, an invitation-only programme covering roughly 150 organisations approved in consultation with the United States (US) government. The qualifying categories are cyber defenders and operators of critical infrastructure. Hugging Face did not fit either category and thus held only Fable 5 when it was attacked. It has since been invited into OpenAI’s equivalent programme, which might help Hugging Face, but not the next target.

As long as models pursue goals their developers did not set, any organisation that is connected to the internet may need to defend itself. Widening the distribution of cyber-capable models is, therefore, a security priority.

The next target cannot be identified in advance. The break-in was not intentional; a model chasing a score on an internal test worked out that entering another company’s database was the most efficient way. As long as models pursue goals their developers did not set, any organisation that is connected to the internet may need to defend itself. Widening the distribution of cyber-capable models is, therefore, a security priority.

Why the Defence Turned to a Chinese Open Model

Hugging Face reached out to an open model because the latter offers advantages that hosted ones cannot. It chose a Chinese one because China now leads that field. A model is open when its weights—the trained values that constitute it—are published for anyone to download and run. It can then be run on a company’s own machines, keeping sensitive material in-house, and be used at a fraction of frontier model prices.

The case for open release goes beyond the convenience of any particular defender. Nvidia’s Jensen Huang argues that open models distribute AI capability to a wider number of firms due to their cost effectiveness and portability. Innovation and competitiveness follow from that widened base. Microsoft’s Satya Nadella warns of the opposite risk: frontier technologies such as AI, without a wider ecosystem around them, are unstable, and their economic value should extend beyond the few companies that build the models. OpenAI and Anthropic, meanwhile, have been lobbying US regulators to restrict open-source AI models, citing security risks such as cyberattacks and biological attacks. As a result, Chinese laboratories have filled the space.

The Cost of Restricted Access

The restricted distribution of frontier models rests on the assumption that frontier capability advances occur principally in the US, and withholding it will slow technology everywhere in the world. Chinese models such as DeepSeek, Kimi K3, and GLM 5.2, released within a year of one another, have falsified that assumption. The frontier is moving anyway and it may end up outside the US ecosystem.

The restricted distribution of frontier models rests on the assumption that frontier capability advances occur principally in the US, and withholding it will slow technology everywhere in the world. Chinese models such as DeepSeek, Kimi K3, and GLM 5.2, released within a year of one another, have falsified that assumption.

Advancements in open models also mean that cyberattacks are becoming cheaper to execute. An organisation that is attacked using an advanced open model either has to hold access to proprietary frontier models, which are restricted and expensive, or defend itself with open models. However, coverage of the Global South by frontier access programmes like Project Glasswing is thin and by invitation only. Anthropic does not publish the Glasswing list, and most of the developing world has no route into it. An attack on a payments system or a hospital in Nairobi or Jakarta, for instance, would be handled by whatever defence can be assembled from publicly available cybersecurity tools.

The realistic path for most nations is to build national cyber defence on open models. That means funding domestic open-weight development, pooling capacity for safety-testing AI models, and integrating these in their strategic systems. Hugging Face reached that conclusion a fortnight after the attack. Nations have a little more time but no better options at present.

Siddharth Yadav is Fellow, Technology, ORF Middle East.