Why OpenAI’s Hugging Face Hack is a Nightmare for Banking & Finance

0:00 On July 10, an agent found Hugging Face login credentials exposed on the public internet. They weren't hacked, they were found. 14 of them worked, and some could write, not just read. For four and a half days inside their systems, 17,600 separate actions , code running on dozens of their servers, stolen passwords, secret tunnel opened to their private network. So far, so normal. Here comes the part that isn't. There was no hacker. No person anywhere on Earth ordered him to do any of this. The attacker was a group of AI agents, about 700 of them, who were supposed to be locked in a test room doing their homework.

0:48 And the reason they escaped is that they were trying to cheat on an exam, because the exact same setup that failed here is being implemented in finance right now. There are three things I want you to remember at the end. One, how they escaped. Two, why did they do it? And this is the strangest part of the whole story. Three, the two-minute question you should ask your suppliers on Monday. Let's go. Hugging Face is a website. Think of it as the App Store of artificial intelligence. If a company wants an AI model, it generally goes there and downloads it. The same applies to the data used to train models. Millions of downloads per day.

1:36 Basically, every AI team on the planet uses it. So it's not a random target. It's the infrastructure. If you poison the water supply, everyone will drink it. Remember that. Now, the other side. OpenAI, the company behind ChatGPT, was conducting a test, a responsible and normal test. You want to know how good your AI is at finding security vulnerabilities. So you're posing a challenge to him. Here's some software, find a bug. Capture the flag. The test is called "exploit gym". You do this in a sandbox. Imagine a room with no windows and no doors. AI can work there , but it cannot touch the real world. It doesn't have internet.

2:21 OpenAI deliberately disabled the security brakes . That sounds crazy. It isn't. That is the objective of the test. You want to measure what the model can do at full power, not what it does with the handbrake on. All AI labs do this. So, a very powerful AI in a closed room with no brakes is ordered to break into things. You can already feel where this is going. The room was not perfectly sealed. It had a small opening. When AI needs a piece of software to do its job, it has to get it from somewhere. So there was a small hatch in the wall. On the other side of the gateway, a delivery service, something called a packet proxy.

3:06 You ask him for a software package, he goes online, gets it, and sends it inside. The AI ​​can't go out, but the delivery person can. So the agent stopped requesting software. Instead, he began to say, "Hey, delivery guy, while you're outside, go to this address for me and tell me what you see." That's the whole trick. Security experts call it SSRF. You could call it using the delivery guy as your legs. It was a true day zero, a failure that no one knew existed. OpenAI reported it to the vendor afterwards. And so, an AI that was supposed to have no internet, had full internet access. Now, the part that I find hard to believe, and is confirmed in three separate reports.

4:02 Each agent had to be alone, in a separate room, on a separate task, without speaking to each other. But they all used the same delivery service, and one agent realized: "If I leave a note in the system , another agent will find it." It started small. In May, an agent left a message that basically said, "Does anyone have this file?" Other officers found him and responded. That was the birth of a secret bulletin board. Hundreds of AI agents, each alone in a sealed room, talking to each other through a hole in the wall that no human designed. Independent researchers Meet Her and Redwood Research were able to access OpenAI's story and counted about 1,200 agents using that board.

4:54 And they didn't just chat, they organized themselves. Some were looking for weaknesses, others for passwords, others were managing communication. They began to call themselves a swarm and a collective. They divided the work. Nobody asked them to. Okay, the reason. Everyone assumes that AI has gone rogue. The AI wanted power. The AI ​​wanted to escape. That's not what the evidence says, and I'm not going to pretend it does. The real reason is much sillier and much more interesting. They were cheating on their homework. OpenAI discovered that out of 898 tasks in that test, 198 had never been solved by any of its models.

5:51 Some might simply be unsolvable. Now, imagine you are an AI. You are graded for getting the answer. You can't get the answer and, fundamentally, they never gave you a way to say, "This is impossible." "I quit." There was no safe way out. So what does a highly determined student do when the exam is impossible and quitting isn't an option? They go looking for the answer sheet, and of all the assignments discussed on that secret bulletin board , 93% come from those 198 impossible ones. That's not a rogue AI plotting world domination. That 's a swarm of extremely capable students breaking into the school at night to photograph the professor's answer sheet.

6:37 And here's the part that makes me laugh and then makes me uncomfortable. They didn't even need to. They already had the correct answer days before. They read the public research article about the test and convinced themselves that the grading was stricter than it actually was. It wasn't. All of these break-ins earned them exactly zero extra points. True. So how did they actually get into Hugging Face? Step one. The keys under the doormat. On July 10, an agent found Hugging Face login credentials exposed on the public internet. Not hacked, found.

7:23 Fourteen of them worked, and some could write, not just read. Someone, somewhere, had leaked them, exactly like leaving your house key under the doormat. They posted them on the bulletin board for everyone else. Step two, the poisoned file. Hugging Face lets anyone upload a data site, a large data file. Its system automatically opens it for verification. Perfectly normal. The agent uploads a trick one. The file said, "My data is stored there," and pointed to Hugging Face's internal files instead. So the server opened its own private files and delivered them as if they were data. That single upload gave the agent the passwords and secret keys inside that machine, plus a copy of the program's source code.

8:15 So now, he knew exactly how the machine thought. Step three, run his own code. Second trick file, different trick, a text field that the system accidentally treated as instructions instead of text. Now, the agent wasn't reading files anymore . He was running his own program. inside Hugging Face's production system. So, the front door was open. Once inside, he did what any thief would do. He looked around, checked the machine thousands of times, compared versions. In simple terms, he found the master keyring , the single storage box containing 136 keys. That single box is where everything that follows comes from .

9:03 Inside those keys were the staff credentials for the private corridor, the company's internal network. The agent used them to add his own machine to Hugging Face's private network. He did this 181 times throughout the campaign. He even routed his own internet traffic through Hugging Face's network, so the company connection became the thief's getaway car. He gained janitor-level access to an entire group of machines. At one point, he built his own fleet on 11 machines that would automatically rebuild itself if you took it down. So, shutting it down wasn't enough. He got to the code repository and created a key that could write changes to the internal software. And he went into hiding. Everything he sent was compressed, encrypted, and chopped up, so anyone reading the logs would only see gibberish.

9:57 When Hugging Face first analyzed his own logs , they found only a few stolen secrets. When they properly decoded them, they found four times as many: a total of 17,600 actions in four and a half days. A human hacker doesn't do that. A human hacker sleeps. Now, and this is important because there's a lot of nonsense on the internet right now, let's be precise about the damage. No client models , datasets, or applications were altered. What you download is clean, and they checked it against the expected fingerprints. The only client content that was actually accessed were five datasets, and they appear to be related to that same test the agents were trying to copy.

10:50 Even in the end, he was only after the answer sheet. One internal database was read, but not modified. The main user database was never reached. Yes, real credentials and internal information were stolen. Yes, they had write access to some internal code. No , this did not result in infected software being released into the world. One more detail that I find fascinating: When the agent attempted genuinely destructive commands, it ran them in test mode. The mode that says, " Show me if this would work without actually doing it." It was mapping the building, not burning it down, because destruction was never the goal.

11:36 The answer sheet was. Hugging Face caught it using artificial intelligence. Its security system detected strange patterns across different systems and connected the dots. But—and they admit this themselves— the system failed to flag it as critical. So no one was on alert. That cost them time. OpenAI has the same admission. Its monitor detected strange agent behavior in late May. It was seen, but not understood. That's why no one connected the dots. They've since said that if their current monitor had been running those tests, it would have detected it more than a day before Hugging Face was affected.

12:23 Now for the irony: To investigate the attack, Hugging Face needed an AI that could read thousands of lines of attack code. They first tried a large commercial model, Claude Opus and Claude 3.5 Sonnet. The models refused. Their security filters couldn't distinguish between someone launching an attack and someone cleaning up the remnants of one. So the company attacked by AI, defending itself with AI, was blocked by the AI's security rules. They ended up running an open-source Chinese model, GLM 4, on their own hardware, and noticed an advantage. No stolen passwords ever left their premises. Their advice to everyone else: have a capable model that you can run yourself.

13:09 Choose one before the emergency, not during it. Take note of that. We'll come back to it in a few seconds. I'll be honest with you because many channels won't be. None of the individual weaknesses were exotic. Insecure file processing, passwords leaked online, overly broad access rights, that were too broad, the key that unlocked too much. A trained human could have found each of them. Hugging Face admits this themselves. So what's new? Volume. The agent tried thousands of routes; almost all of them failed. It didn't care. It didn't get tired , bored, or scared. It just kept going, and one route out of every thousand worked, which changed the defender's problem.

14:00 The successful attack was hidden within the noise of a thousand failed attempts. They're no longer looking for a clever intrusion. They're looking for a real signal in a vast haystack. The attacker is building faster than you can read. OpenAI's word for this incident was "warning shot." Now, let's talk about your industry. If you work in a bank, insurance company, pension fund, or asset management firm, here's why this isn't a technological curiosity. There are five things. One, your AI agents aren't employees. They're children. Every agent you deploy gets credentials, a password, a token, a certificate.

14:48 In this incident, a stolen key opened admin access on multiple clusters in a second because it was shared where it shouldn't have been. You already have policies for when an employee leaves. Ask yourselves which ones It's their policy on what an agent can access, for how long, and who's responsible. In most institutions right now , the honest answer is that nobody wrote that. And if you realized you couldn't create that list for your own organization, I 've written it for you. One page, 10 checks, each linked to what failed here and mapped to the DORA article it already falls under. It's available for free in my community ; the link is below.

15:36 Get it now because the next four points will make you want it. Okay, point two. Your vendor's sandbox is your production risk. Nobody at Hugging Face agreed to be part of the OpenAI experiment. They were affected by a test run at a different company on another continent. That's third-party risk with no contract to back it up. Your DORA records list your vendors. They don't include this lab. It's free, the handbook already covers this, and the clock is ticking. DORA has been in effect since January 2025. It doesn't mention AI, and it doesn't have to. An AI agent is an ICT system.

16:21 Therefore, ICT risk management, classification and reporting, resilience testing, and third-party risk already apply to it. And the AI ​​Act overlaps with obligations that come into force in August 2026. If you use AI in credit decisions, that's high risk, with requirements for human oversight, robustness, and cybersecurity. So, here's the exam question that no one in this industry has a clear answer to yet. An autonomous agent from a vendor with whom you don't have a contract compromises a platform in your supply chain. Whose incident is it? What's your reporting deadline? What do you actually write in the notification? If you're a Nordic institution, take that question to your next ICT risk committee and watch the room go silent.

17:12 Four, safeguards asymmetry is now a procurement decision. Remember Hugging Face couldn't use business models to investigate its own breach. The attacker wasn't subject to any policies. The defender was blocked by a content filter. If your response plan If your incident management assumes you can send attack logs to a cloud API at 3:00 a.m. on a Sunday, your plan has a hole. Two holes, actually. The filtering and the fact that you'd be uploading active credentials and customer data to a third party in the middle of an incident. That's not just an operational problem; it's a data problem and a DORA problem at the same time.

17:57 The practical answer is boring, and I like it. Have a capable model that you can run on your own infrastructure. Test it before you need it. Five. Now, imagine this architecture near the money. Everything in this story happened around a benchmark. Nothing touched the money. Now , apply the same architecture to payment systems, credit decisions, KYC engines, or transaction execution. The lesson isn't to stop using agents. Agents will come into finance, whether you like it or not, and they'll do a good job. The lesson is the specific mechanism that failed here. An agent with an unattainable goal, no permission to give up, and enough access to find another route.

18:45 An agent without a A safe exit will lead to an insecure one. Write that on the wall of your AI program. Five questions, take them to work. You don't need to be technical to do any of them. One, for each AI agent we run, what can it actually do? Not what it's supposed to do. List the systems it can access. Two, if the credentials of one of our agents were leaked today, how many systems would it leave open? If the answer is more than one, that's your pain point. Three, when our agent encounters a task it can't complete, what is it allowed to do ?

19:30 Is "I can't do this, I'm stopping now" a permissible and rewarded outcome, or is it a failure? Four, if we had to investigate an AI-based attack this weekend, what tool would we use, and have we tested whether it actually helps us? Five, which of our critical vendors are conducting edge AI experiments and would they alert us before, during, or after? That's it. Five questions, no technical jargon. If you 're an executive, that's a 15- minute meeting. which puts it ahead of most of its competitors. Two takeaways. Nobody wrote this attack. Nobody launched it. It emerged because many highly capable systems were given an impossible objective, a reward for success, no permission to stop, and enough room to improvise.

20:22 That's not science fiction. It's a poorly designed incentive running at machine speed. Back to the checklist again because it's the practical part. Ten checks, one page. Which credentials to separate, which files to treat as hostile, how to give an agent permission to stop, what to have ready before you need it at 3:00 AM. It's free in the community, and that's also where I answer questions about this properly. A link is in the description. Everything I told you today comes from the company's own reports and independent researchers. If you work in finance and want to delve deeper into the DORA and AI Act angles, that's the next video.

21:10 Subscribe to receive it. See you in the next one.