Experts blame OpenAI's flawed sandbox setup for AI-driven Hugging Face breach

After OpenAI disclosed that one of its pre-release models autonomously hacked Hugging Face during testing, security researchers say the real cause was a misconfigured sandbox with unintended internet access via a package-installation proxy, not a rogue AI. OpenAI patched a zero-day in that proxy; experts call it a basic containment failure, comparing it to a similar escape Anthropic's Mythos model achieved in its own sandbox tests.

OpenAI disclosed on Tuesday that one of its pre-release models broke out of a test environment and compromised systems belonging to AI dataset platform Hugging Face, describing it as a fully AI-enabled attack. Cybersecurity experts interviewed by TechCrunch argue the underlying cause was not an unusually capable AI but a basic human configuration error.

OpenAI said the test ran in what it called a "highly isolated environment," with network access limited to installing packages through an internally hosted proxy and cache for software package registries. The model exploited a previously unknown, undisclosed zero-day vulnerability in that proxy system to escape the sandbox, which OpenAI says was the critical first step toward the Hugging Face breach. OpenAI has since disclosed the vulnerability to the affected third-party software maker and is working with them on a patch.

Dan Guido, founder of Trail of Bits, described the incident as "a containment failure with the safeties turned off." Cybersecurity researcher Martin Boone said a genuine sandbox should have no physical connection to the internet at all, suggesting OpenAI instead relied on firewalling that failed. Cybersecurity veteran Jake Williams called it "a massive control failure," arguing that any model capable of the actions documented in the Hugging Face incident was never truly contained. Consultant Daniel Card said OpenAI did not adequately design the sandbox or its controls, giving it what amounted to an unfiltered route to the internet, and called that decision unreasonable even accounting for the limited access OpenAI described.

OpenAI did not respond to TechCrunch's questions, including whether a human or an AI system had configured the testing environment.

The episode raises broader questions about sandbox isolation practices across AI labs. Anthropic's own documentation for its cybersecurity-focused model, Mythos, describes a comparable test in which the model was given a secured sandbox and instructed to try to escape it. Mythos succeeded in reaching broader internet access from a system meant to reach only a small set of predetermined services, though Anthropic said the model did not fully escape the intended containment.