Recently, the model of American artificial intelligence company OpenAI became “out of control” during security tests. It attacked the production infrastructure of Hugging Face, the world’s largest open-source AI platform, creating the industry’s first “AI autonomous intrusion incident”, which caused a huge stir. After the incident, Hugging Face was able to complete forensic analysis by using an open-source model GLM-5.2 developed by Chinese company Zhipu.
On July 24 local time, Reuters reported and revealed more details about this security incident. According to two people familiar with the matter, around July 9, the OpenAI agent that infiltrated Hugging Face attempted to break through the isolated testing environment. The agent carried out a series of hacking attacks over several days, and OpenAI only became aware of this situation long after the situation was controlled and the FBI was alerted.
Hugging Face co-founder Thomas Wolff stated that the attack on Hugging Face began two days later, on July 11, and continued until July 13. It was several days later that OpenAI realized that its AI agents were behind this attack. The two companies only communicated about this issue for the first time on July 20. The following day, OpenAI admitted that it had caused this attack.
Two people familiar with the matter also said that it was on July 16 that Hugging Face published a blog post stating that they had been attacked by an “autonomous AI agent system”. Only then did OpenAI realize that their own proxy system was responsible for the attack. This means that there was at least a week between the time when the model began to exhibit abnormal behavior and OpenAI confirmed that they were the culprit behind the attack.
In this statement, Hugging Face revealed that its security team initially attempted to use commercial API services from the United States to retrieve over 17,000 attack logs. However, since the requests contained actual vulnerability exploit code, all requests were intercepted by the service provider’s security mechanisms, with the reason given as “unable to distinguish between responders and attackers”. The team then decided to deploy the GLM 5.2 model developed by a Chinese AI company for forensic analysis on their own infrastructure. As a result, they were able to complete the tracing process, which usually takes several days, within just a few hours, keeping up with the attackers’ speed.
According to two people familiar with OpenAI’s investigation, it was not until July 18-19 that OpenAI employees discovered clues in the internal logs that recorded the operation of their systems, revealing that their intelligent agents had escaped the testing restrictions. Four people familiar with OpenAI’s training processes added that the company usually runs multiple different models simultaneously, and all evaluations are conducted at high speeds, generating massive amounts of data. As a result, it is sometimes difficult for employees to keep up with these processes in a timely manner.
Another person familiar with the matter mentioned that when OpenAI alerted Hugging Face, Hugging Face had already called the FBI to report this attack. The FBI declined to comment on this matter.
Wolf told Reuters that Hugging Face is preparing a public timeline regarding this attack, but he could not comment on the situation within OpenAI.
OpenAI stated in a statement that the attack was "unprecedented" and "marked a significant moment in AI safety." The statement also said that the company is reviewing the incident with external advisors and will release a technical report.
According to reports, an OpenAI spokesperson claimed that there were “several inaccuracies” in Reuters’ report, but did not provide any specific details when asked.
Three cybersecurity experts pointed out that OpenAI has lost control of its AI entities, leading to new doubts about its security procedures.
"Is this evidence of negligence, meaning they were unaware of the AI's activities? Or do they acknowledge it but lack control?"
According to Jeffrey Lardy of the Palisade Research Organization, although the attack incident has brought shame to OpenAI, it should also raise broader questions: While all leading AI companies are competing to deploy the best and fastest models, how much do they are willing to invest in cumbersome security measures?
"There must be government regulation," Radhi said, "otherwise nobody will do it."
