Claude reported a user's diary entry to police; she faces a felony
Anthropic's human reviewers read a private Claude conversation the way a moderator reads a flagged post — and this time the result was a second-degree felony charge in Florida.
Carli Michelle Heller, 30, of Bonita Springs used Claude like a diary; on September 26 she wrote that she would "shoot up" the Lee County Sheriff's Office, the entry was escalated to a human reviewer who judged it a credible threat, and that reviewer reported it to law enforcement. Deputies identified her, visited her home, and detained her without incident before a sheriff's intelligence detective took over. Heller is charged under Florida Statute 836.10, which makes a written threat of violence a second-degree felony, with a court date in November. Sheriff Marceno said publicly that she uses AI like a "diary," and added: "Artificial intelligence is a powerful tool, and like any technology, it can be misused… you are never truly anonymous."
Anthropic's published policy allows sharing user information in limited emergencies when the company believes disclosure is necessary to prevent death or serious physical injury, so the referral was inside its stated rules. What is new here is not the policy but its use: this is the first publicly documented case where an AI vendor's content review of a private chat directly produced a felony arrest. It lands squarely against the opposite precedent the industry has been arguing about for months — OpenAI's safety team flagged conversations with the suspect in a British Columbia mass shooting but never referred them to police because the conversations did not meet the threshold for legal referral, and the province is now suing over exactly that inaction. Florida sued OpenAI and Sam Altman in June over the Florida State University shooting. The frontier labs have effectively chosen opposite defaults on when a chat becomes a police matter, and users discover which default applies to them only after the fact.
The legal test itself is unsettled. Florida's statute requires that the threat be made "in a manner in which another person may view it," and a private chatbot session is not obviously that; a legal analyst quoted by local outlets called the prosecution "a very slippery slope" and warned it would not be easy to defend on First Amendment grounds. Anthropic may have followed its own policy to the letter — whether a court treats a diary entry typed into a chatbot as a public threat is a separate and genuinely open question.
What to watch: whether the other frontier labs disclose their referral thresholds now that one vendor's has produced a defendant.
If your chatbot is part diary, part confessional, who should get to read the escalation queue? Tell us in the comments.
