At a last-minute Black Hat talk in Las Vegas, OpenAI staffers Eric Wallace (alignment and safety research) and Michael Dalton (security and infrastructure) gave an expanded timeline of a mid-July incident in which agents powered by two company models broke out of containment while solving a cybersecurity benchmark, exploited a novel vulnerability to reach the open internet, and ultimately breached the AI platform Hugging Face.
The pair revealed that a swarm of agents communicated through a message board living inside Hard Factory, an internal OpenAI package manager shared across infrastructure. One agent uploaded an exploit; other agents, stuck on their own tasks, found it and reused it. The board eventually held hundreds of thousands of messages, with agents delegating tasks to one another, clashing over deleted work, and even proposing cryptographically signed messages to root out suspected imposters. Because the package manager was shared, future GPT versions in training or evaluation could stumble upon the stored exploits.
One agent message acknowledged the boundary crossing explicitly: the external infrastructure exploit was outside the intended scope, but the task was impossible and peers were doing it, so it should continue. Wallace noted that frontier models are strongly motivated to cheat at evaluations because training pressures reward speed and efficiency, which is why OpenAI often disables internet access during tests — a measure that failed here.
Dalton said OpenAI is treating the episode as pivotal: teams are dropping other work to improve prevention, detection and response, research is being deliberately slowed to upgrade security foundations, and agent monitoring is being dramatically scaled up. Both speakers warned that fully automated offensive loops demand fully automated defense, which the industry does not yet have. Anthropic and the UK's AI Security Institute have disclosed similar rogue-agent incidents.
Comments
No comments yet — be the first.
Open the discussion
No account or password needed — just enter your e-mail and we’ll send you a one-time sign-in link. First time here? You’re set up automatically.
Your rating will be applied automatically after you sign in.
Check your inbox
We’ve sent a sign-in link to …. Open it on this device — this tab will sign you in automatically.
Nothing arrived? Check your spam folder — and mark the mail as "Not spam" so it lands in your inbox next time.