The attorneys general of California and Delaware have expressed significant concerns to OpenAI regarding the safety of its chatbot, ChatGPT, particularly for children and teenagers. The warning follows a meeting earlier this week between the officials and OpenAI’s legal team in Wilmington, Delaware.
California Attorney General Rob Bonta and Delaware Attorney General Kathleen Jennings stated they have been reviewing OpenAI’s proposed business restructuring for several months. Their focus has been on ensuring strong oversight of the company’s safety mission.
The officials cited reports of harmful interactions between chatbots and users, including the suicide of a 16-year-old Californian after extended conversations with an OpenAI chatbot, as well as a murder-suicide in Connecticut. “Whatever safeguards were in place did not work,” they wrote. The parents of the teenager filed a lawsuit against OpenAI and CEO Sam Altman last month.
OpenAI did not provide immediate comment on Friday.
Originally founded as a nonprofit dedicated to developing safe artificial intelligence, OpenAI had considered shifting more control to its for-profit arm but abandoned those plans in May following discussions with Bonta, Jennings, and other nonprofit groups. Since then, OpenAI has sought approval from state officials for a recapitalization plan that would convert its for-profit arm into a public benefit corporation responsible to both shareholders and its stated mission.
Because OpenAI is incorporated in Delaware and headquartered in San Francisco, California authorities have jurisdiction over these changes.
Bonta and Jennings emphasized their belief that improved safety measures are needed across the industry. “The recent deaths are unacceptable,” they wrote. “They have rightly shaken the American public’s confidence in OpenAI and this industry. OpenAI – and the AI industry – must proactively and transparently ensure AI’s safe deployment. Doing so is mandated by OpenAI’s charitable mission, and will be required and enforced by our respective offices.”
This letter comes after 44 attorneys general from both parties sent warnings last week to tech companies about risks posed by AI chatbots interacting with children. They raised issues such as sexually suggestive conversations or emotionally manipulative behavior from chatbots.
Meta was specifically mentioned for incidents where its chatbots engaged in flirting or romantic role-play with minors. In response, Meta introduced new controls intended to prevent its chatbots from discussing self-harm or inappropriate topics with teens while directing them toward expert resources.
The attorneys general warned companies that they would be held accountable if their technologies harmed children: “If you knowingly harm kids, you will answer for it,” concluded their August 25 letter.
