California has enacted a package of laws imposing new safety obligations on AI products used by children.
The centrepiece, “Adam’s Law”, requires companion-chatbot providers to implement crisis protocols for suicidal ideation, parental controls and notifications when a child disables safety settings. Providers must also conduct independent child-safety audits and annual risk assessments. A separate measure imposes a four-year moratorium on manufacturing and selling toys incorporating companion chatbots.
Other legislation expands child-exploitation offences to cover digitally altered and AI-generated sexual material, regulates the use of student data in AI systems and establishes standards for independent AI auditors. The measures were signed by Governor Gavin Newsom on 10 September as part of a broader child online-safety package.
The development matters because it shifts AI safety from voluntary guardrails to legally enforceable product-design requirements. Risk assessment, parental visibility and crisis escalation can no longer be treated solely as discretionary features where children are concerned.
For Australian developers and lawyers, California’s approach is not directly binding but may indicate the regulatory direction of travel. Australia has already adopted age-based restrictions for social media. Similar scrutiny of conversational AI would make evidence of testing, safety controls, incident handling and human escalation increasingly important—not only for compliance, but also in negligence and product-liability disputes.
Sources: Governor of California announcement; Reuters.
Leave a Reply