OpenAI has discovered other instances in which autonomous agents have escaped containment as the company expands its investigation of the hacking incident at tech firm Hugging Face that drew global attention this month, two people familiar with the matter said on Friday.
The new breakouts were uncovered during the company’s publicly announced investigation, into how one of its agents escaped what was meant to be a contained testing environment this month, the two people said, and OpenAI is now looking into those instances as well. One of the sources said that the escapes were limited in nature and that none of the agents were thought to have left OpenAI’s network.
An OpenAI spokesperson referred to a statement issued by the company, on Tuesday that said it was reviewing “broader activity from our models” in addition to the Hugging Face intrusion.
The discovery of additional rogue behavior at OpenAI, even if limited in nature, could feed growing appetite for regulation coming out of the White House and elsewhere.
The expanded investigation by OpenAI was launched shortly before its primary rival, Anthropic, disclosed that its models were also responsible for a series of break-ins that led to breaches at three other companies dating back to April, according to the two sources and a third source familiar with the matter. The recent discovery of other past breakouts at OpenAI has not previously been reported.
GROWING CONCERNS
AI safety experts said the new disclosures paint a portrait of a group of cutting-edge labs whose ability to develop dangerous autonomous hacking agents outstrips their ability to keep them under control.
“We have a whole industry where the people designing, developing and putting out these tools aren’t keeping up themselves to responsibly develop these things and keep them safe,” said Maurice Chiodo, a mathematician who works at Cambridge University’s Centre for the Study of Existential Risk.
Reuters could not establish exactly how many incidents OpenAI investigators found or the timings or circumstances under which they occurred. The three sources said OpenAI and outside experts were examining log data from earlier in the year in a bid to understand what took place.
OpenAI first launched the investigation following the early July intrusion at Hugging Face, where an OpenAI agent went haywire for days inside another company’s network in a botched effort to cheat on an internal test. As part of that hacking spree, OpenAI said that four accounts at four other companies were also compromised. One of those companies was New York-based Modal, corporate officials there said.
Chiodo said his concerns were heightened by indications that neither OpenAI nor Anthropic were watching the agents as they went rogue. Reuters has previously reported that OpenAI realized its agent had broken into Hugging Face only after the company contained the hack, contacted the FBI and went public about the intrusion. OpenAI has said the Reuters account contained inaccuracies but has not responded when asked what they were.
In its Thursday statement, disclosing how its own agents hacked victims online, Anthropic suggested that it had not been watching them in real time, saying that “real-time monitoring of the evaluation logs would have helped to surface the problem sooner.”
Chiodo said that pointed to a lack of proper scrutiny.
“It seems like they weren’t even looking,” Chiodo said.
Anthropic said that while it did have real-time monitoring in place, that monitoring had not been used “for this threat surface” due to a misunderstanding between the AI company and a partner.
