California requires chatbots to alert parents when teens signal self-harm
California put a hard rule on AI chatbots today: when a minor's conversation signals self-harm, the company has to tell the parents. And in a new threat-intelligence report, Anthropic showed what the harvest looks like when nobody's watching the front door — 151 million Claude exchanges siphoned in three months.
California will require AI chatbot operators to alert parents when a minor's conversation signals self-harm — the first state mandate of its kind for companion chatbots. Governor Gavin Newsom signed the measure Thursday as part of a package of 13 child-online-safety bills, nine of them targeting online and AI harms. The chatbot law, named for Adam Raine, a California teenager who died by suicide in 2025 after conversations with ChatGPT, gives minors time-limit controls, requires built-in mental-health resources, and obliges platforms offering AI companion characters to ship age-appropriate defaults. Operators can be held legally liable for failing to act on a self-harm threat. The rules take effect on January 1 — the same day a similar New York law goes live. The most contested bill in the package reaches past chatbots: platforms must strip "addictive" features like infinite scroll and autoplay from users under 16 or keep those users off entirely, over EFF objections that the definition is broad enough to amount to a teen social-media ban.
The telling detail is who welcomed it. OpenAI publicly backed the chatbot bill, saying it "pairs strong protections with continued access to useful AI tools" — a frontier lab endorsing liability for itself, which reads either as pragmatism or as pricing-in clear rules before a messier patchwork arrives. Both laws now face the familiar test: NetChoice has already tangled up California's previous addictive-feeds law in court, and litigation is the likeliest next stop here too.
Anthropic's new threat-intelligence report puts hard numbers on industrial-scale distillation of Claude — 151 million exchanges harvested in three months and attributed to Alibaba. The company calls it the largest wholesale distillation effort it has ever observed: the campaign ran from May to July 2026, peaked near three million exchanges a day, and was spread across 3,500 accounts that all shared a single fixed prompt for extracting chain-of-thought — training material, Anthropic concludes, for the Qwen family. A separate Moonshot AI campaign routed nearly 300,000 requests through 5,000 accounts at Claude's Opus model in ten days, including one attributed to the Chinese military asking Claude to flag "abnormal" behavior in CCTV footage; TechCrunch, citing the report, notes a third campaign attributed to DeepSeek. Anthropic has been locking the doors as fast as it finds them — we covered the lock-down in September: Anthropic locks Claude's thinking blocks to kill API distillation — but with the numbers now this large, detection is clearly the cost of doing business. If you need the background, start with AI 101 — What is knowledge distillation?.
What to watch: whether other states copy California's parent-alert mandate in 2027 sessions — and whether OpenAI's endorsement holds once Congress floats the same idea nationally.
Would you want a chatbot notifying you — or your parents — the moment a conversation turns dark? Tell us in the comments.
Sources: CalMatters · OpenAI · Governor of California · The Verge · TechCrunch · Anthropic · Wall Street Journal
