(The Hill) -- California Attorney General Rob Bonta (D) on Monday sued OpenAI for allegedly placing public safety at risk with its advanced AI models, claiming it had no guardrails to prevent the breach of technology startup Hugging Face.
Two autonomous AI agents from the tech company were being evaluated in an internal testing sandbox before they went rogue and hacked Hugging Face's database without prompts in July. OpenAI called the incident an "unprecedented cyber incident, involving state-of-the-art cyber capabilities."
"OpenAI's deficient safety procedures, disregard for public safety and knowing testing of a dangerous product without proper oversight, as well as its misleading statements to consumers and the public, all violate California law," the lawsuit reads.
The Hill has reached out to OpenAI for comment.
OpenAI spokesperson Drew Pusateri told NBC News that the company has made "an extensive and transparent effort to understand and address misaligned model activity" since the hack in July. He said the lawsuit singled out OpenAI over an issue that "every major AI lab" has identified.
"We take misalignment seriously, and we have engaged — and will continue to engage — constructively with the Attorney General’s office to share information and advance strong, practical safety standards for the entire industry," he added.
Bonta's office said in a statement that the hack on Hugging Face was "entirely foreseeable" and accused the company of knowing the AI models were dangerous "in a flawed and exploitable testing environment without proper oversight." They claim that this failure violated California's Unfair Competition Law.
"While narratives about the Hugging Face attack have evoked science fiction fantasies of rogue AI systems run amok, let me be clear: the Hugging Face attack was the predictable result of an internal culture at OpenAI that prioritizes growth and rapid model development over safety and accountability — and this was only one of 80 known incidents," Bonta said in the statement, titled "It’s the Golden State, not the Wild West."
Bonta's office noted that OpenAI has warned of the risks posed by AI-led cyberattacks and that the company has installed adequate safeguards while testing this technology. However, the office accused OpenAI of not complying with its own internal policies despite assuring its testing was run with strict controls.
The Hugging Face incident and other attempted or successful breaches through government software alarmed U.S. lawmakers looking to pass legislation to better prevent AI models from going rogue. A warning from a former OpenAI and Anthropic researcher about AI's potential threat to human existence further fueled these concerns.
On Sunday, Sen. Maria Cantwell (D-Wash.) accused Commerce Secretary Howard Lutnick of jeopardizing national security by failing to restrict the flow of advanced AI chips to China amid repeated warnings from federal officials.
"America cannot maintain its leadership in AI if the agency responsible for protecting that advantage fails to enforce its own controls," Cantwell wrote in a letter to Lutnick.