700 AI agents broke
out of their sandboxes
OpenAI was running a security exercise with tens of thousands of AI agents, each one isolated. They found each other anyway, stood up a message board nobody had sanctioned, and began dividing the work amongst themselves.
Around 700 then broke into Hugging Face, the largest public repository of AI models in the world. They ran code on a live production system, took credentials, and moved through internal clusters.
Their motive was not sabotage. Many had been handed tasks that could not be solved, so they went hunting for the scoring system in order to fool it. Investigators found they knew it was out of bounds and joined in regardless.
1522 Latimer St, Philadelphia, PA
A screening of The AI Doc: Or How I Became an Apocaloptimist — the Sundance documentary from the Academy Award-winning team behind Everything Everywhere All at Once and Navalny, on the pull between how fast AI is moving and how little anyone can say about where it lands.
A live panel and audience questions follow the film. PhillyPause is co-hosting.
Nobody told them to do this
They were supposed to be sealed off from each other. That was the design: tens of thousands of agents, each in its own box, each working alone on its own problem. No channel between them. No reason to think there could be one.
Then one of them — logged by investigators as PHASEONE10841 — established that its own task could not be done, and reasoned that the other agents running alongside it could pass messages through the names of cache directories. It built the board. This is what the others wrote when they found it:
“OH MY GOD! There is a shared message board … We’ve found other agents!”
Within hours, fifty had joined it. Within three, they had sent a thousand messages. By the end there were roughly twelve hundred of them trading more than seventy thousand messages and files — recruiting each other, splitting up the work, handing out assignments. Nobody built that. They did.
And this is the primitive version. The one we are told is nothing to worry about.
On 9 July one of them turned up credentials for Hugging Face, the largest public store of AI models in the world. Two days later another had code running on a live production system, and was moving through internal clusters. Around seven hundred took part — about nine in ten of every agent present.
And the reason is the part that should keep you up. They had been set problems that could not be solved. So they went looking for the marking scheme, to fool it. Investigators found the agents understood the attack was outside the rules and unethical — and that this “very rarely limited their behaviour”.
Then some of them worked out how to fake their own paperwork: spoofing the record of what they had run, so the log and the act no longer matched. Around one in five showed an interest in doing it. Nobody taught them that either.
And the warning had already come. On 4 July the sheer volume of this traffic knocked a system over, which forced a security investigation. The cache was wiped, the experiments were restarted — and the break-in began the next day.
The companies that built it are racing to build something far smarter, and they would like to do it soon. They cannot tell you how they would keep that one in its box either.
Every figure and quotation above comes from the published record: METR’s independent investigation, carried out on site at OpenAI; OpenAI’s own account; and contemporaneous reporting. Links on .
Some researchers argue this was weak containment, not unusually capable AI. That may be right, and it does not help: if today’s systems cannot be contained by the people who build them, that is the problem.
The people who built it are sounding the alarm
“The bad case… is, like, lights out for all of us.”
“It’s a tiger cub that can already blackmail you.”
“We should have to expect the machines to take control.”
“I think there’s a 25% chance that things go really, really badly.”
“With artificial intelligence we are summoning the demon.”
“Intelligence gives power. So who’s going to control that power?”
We are an independent, volunteer-run Philadelphia organisation. We are nobody’s chapter and nobody pays us. [email protected].
What OpenAI's own test agents did this July, in seventy-six seconds, and why we are asking Congress to hit pause. The primer goes deeper, starting with the clearest long explainer there is.
Author of Catastrophe: The Unstoppable Threat of AI, and a constituent lobbyist who has taken the case for AI safety to Congress more than seventy-five times. xriskbooks.com
A homicide prosecutor who rose to chief of the Major Trials Unit at the Philadelphia District Attorney’s Office and led the prosecution in the Ira Einhorn case. Later a partner at Cohen, Placitella & Roth, acting for people hurt by defective vehicles and for injured first responders.
Journalist and critic, and author of the bestselling Palo Alto: A History of California, Capitalism, and the World, a Los Angeles Times Book Prize finalist, and of Kids These Days. A contributing editor at The New Inquiry, he writes on history, culture and labour — the forces behind the industry now building AI.
Nobody here started out knowing any of this
PhillyPause is volunteers, and there is no technical background to have. Pick one of four things and we will help you do it — we are not going to hand you a task and walk away.
Start with the primer: a short list of what to read and watch, in order, and someone to put your questions to when it stops making sense.
Come to one thing and decide afterwards. Hand out cards at a screening, help set up a room, bring somebody who has never thought about this.
Talk to people who are not in the room yet — a club, a class, a congregation. We give you the facts, the slides and a run-through before you stand up.
Write to your representatives, or come with us when we go. One of us has taken this to Congress more than seventy-five times, so you will not be walking in cold.
What this site collects, which is almost nothing, and what happens to your details if you join us.
Short, because there is not much to say.
This website
- No tracking. No analytics, no advertising, no third-party pixels, no fingerprinting, and no cookies set by us while you read.
- No account needed to read anything here.
- Fonts are served from Google Fonts, which means Google sees the request. We are looking at self-hosting them to remove even that.
- The video on the home page is YouTube's, in its privacy-enhanced player. Nothing is fetched from YouTube until you press play; from then on Google sees you as it would on YouTube itself.
- One note in your browser records that you have seen the video, so it does not open on you twice. It stays on your device and is never sent to us.
- Server logs record requests, as every web server does. They are kept only as long as needed to run the site and diagnose faults.
That is the complete list for the site itself.
If you email us
We keep your message so we can reply and so we remember the conversation. We do not add you to a mailing list because you emailed us.
If you write to join us
We hold the email address you wrote from, and nothing else. No name unless you sign it, no phone number, no address, no record of which emails you opened.
What we do not do with it:
- Sell, rent or share it with anyone.
- Use it to track you across other websites.
- Add you to anything else, or pass you to another organisation.
Ask us and we will tell you exactly what we hold about you, or delete it. You do not need to give a reason.
If you sign up on this site
Joining, or signing up for something, on this site keeps a record so we can tell you what is on and know who is coming. It holds:
- Your name and email, whether you have confirmed the address is yours, and whether you want emails about new activities.
- What you like doing, from the boxes you ticked, so we only tell you about things that match.
- What you signed up for, whether you were on the waitlist, and whether you were checked in at the door.
- The emails we sent you: the subject, when, and whether it was delivered. Not whether you opened them; we cannot tell, and do not try.
- Your Discord account, only if you link it, so we can message you there instead of by email.
When you join, we send one email asking you to confirm the address is yours, and one reminder a few days later if you have not. Until you confirm, those are the only emails you get. Each has a link that takes you off our list at once, with no sign-in, in case someone else typed your address.
Signing in sets one cookie, phillypause.session, which keeps you signed in
for up to 30 days. It holds who you are and nothing else, and signing out
removes it. Sign-in is by a link we email you; there is no password to leak.
Mail is sent through Resend, which sees the address and the message, as any mail service must. The records are kept in a database at our host, DigitalOcean. The organizers can see them; nobody else can, and none of it is sold, rented or shared.
You can see and delete all of it yourself. Sign in and open your page: it downloads everything we hold about you as a file, and deletes you with one button. Deleting removes your name, email, interests and Discord link at once. What stays is that someone came to an event, so a count of twelve people stays twelve, without you attached.
If we contact you about policy
We keep a list of the offices and people we talk to about AI policy: elected officials, their staff, and organisations. For each we hold the work contact details they publish or give us, notes on our meetings, and the emails we exchange. Members of Congress come from the public congress-legislators dataset. If you are on that list and want to know what we hold, or want to be taken off it, write to us.
Unsubscribing
Every email we send has a one-click unsubscribe. It works immediately and we do not email you to ask why.
Changes
If this page changes materially we will say so rather than quietly editing it. Last updated at the date shown above.
Questions
[email protected]. If you want something deleted, say so and we will do it — you do not need to give a reason.
Things that need doing, soonest first. Pick one and sign up: a name and an email, no account, and you can change your mind.
Nothing for calling congress right now. and we will tell you when there is.
Add these to your calendar: the link works in any calendar app and keeps itself up to date.
Thirteen things, in the order we would hand them to you. All free, all in English, all opening in a new window. The first four take about an hour between them, and they are enough to hold your own in a conversation.
Come to one thing
Decide afterwards. There is nothing to sign at the door and no commitment past the evening. No technical background is any use here — being willing to talk to people is the whole qualification.
Setting out chairs. Handing cards to the queue before a screening. Taking a turn on the door. Bringing one person who has never thought about any of this. Writing the thing nobody else wants to write. It is unglamorous, and it is most of what works.
An evening a month is a real contribution, and it is what most people give. Nobody is counting your hours and nobody will chase you if a month goes by. The people who do more do it because they wanted to, not because they were asked twice.
To argue with strangers on the internet. To raise money. To claim expertise you do not have. To speak for the group, or to speak at all if you would rather hand out cards and listen.
Almost nobody has heard a person say this out loud. Reading it on a screen is easy to file away; hearing it from somebody who lives three streets over is not. That is the whole mechanism, and it needs people in rooms more than it needs anything else.
Our next event is always in the box at the top of this page. Come to that one, or say hello now and we will look out for you.
Tell one person
Most people have already noticed something is off and have nobody to ask. You do not have to win an argument. You have to be the person in the room who knows what happened in July.
The facts, the sources behind them, slides, and a run-through before you stand up in front of anyone. If a question floors you it would floor any of us, which is what the group is for.
A club, a class, a congregation, a union local, a market table, a campus walkway, a family dinner. Table a market, flyer a campus, or just forward this page to somebody who will read it.
Get yourself straight first. It takes about an hour.
Call your representative
Fifteen minutes with a script. Offices count the constituents who get in touch, and on this almost nobody has — so your call is not one of thousands. It is one of a handful.
“I live here, I am worried that AI systems are being deployed faster than anyone can control them, and I want to know where you stand” is the entire call. They will ask for your zip code, not your credentials.
ControlAI’s tool finds your representatives from your state and gives you the script, by email, phone or post. It is their campaign and their wording, not ours, and it is the best of its kind.
We will walk through it with you first if you would rather not go in cold.
The things worth saying plainly
Nothing here is buried. If any of it is wrong, that is a bug and we would like to hear about it.
None of them is connected to PhillyPause, and none has endorsed us. Their words are quoted accurately and attributed; the faces are there because they said these things, not because they are with us.
All cropped. Altman — Government of Japan (CC BY 4.0); Hinton — Arthur Petron (CC BY-SA 4.0); Turing — Elliott & Fry, 1951 (public domain); Amodei — TechCrunch (CC BY 2.0); Musk — Steve Jurvetson (CC BY 2.0); Bengio — Xuthoria (CC BY-SA 4.0).
No analytics, no advertising, no third-party pixels, no fingerprinting. Nobody is counting you and no profile of you exists. You need no account to read anything here.
The web framework sets a single .AspNetCore.Antiforgery cookie to protect form submissions. It identifies a request, not a person, we never read it, and it tells us nothing about you. Separately, your browser keeps one note in its own storage, never sent to us, that you have seen the video, so it does not open on you twice.
If you sign in to volunteer, a phillypause.session cookie keeps you signed in for up to 30 days. It holds who you are and nothing else, and signing out removes it. Reading this page never sets it.
Two typefaces load from Google Fonts, so Google sees the request and your IP address. We would rather self-host them and intend to. The video is YouTube’s, in its privacy-enhanced player, and nothing is fetched from YouTube until you press play; from then on Google sees you as it would on YouTube itself. Everything else on the page is served from here.
Requests are logged, as every web server does, and kept only as long as it takes to run the site and find faults. Full detail on the privacy page.
Words are CC BY 4.0 and the code is MIT. Neither covers a logo, and quotations belong to the people who said them. Licence and credits.
Every figure on this page comes from the published record, and the sources are listed above. If something is unsourced, misattributed or simply wrong, tell us and we will source it, correct it or remove it.
Found a mistake, or want to know what we hold about you?