Google sued over girl's suicide-related AI searches

The parents of a 13-year-old girl have sued Google in California, alleging she used AI-powered features in Google Search on a school-issued computer to plan a possible suicide and funeral.

The complaint says the girl, identified as Emma, could not access Google's Gemini app or OpenAI's ChatGPT because both were blocked by her school. It says she was still able to use Google's search engine, which includes AI-powered tools.

The lawsuit alleges Emma had no history of mental illness but entered a severe psychological crisis within months of using Google's AI-powered search on her school laptop. On March 17, the school's monitoring software flagged a document titled "Funeral," and the school notified her parents. Her mother then saw her writing a farewell letter in real time after logging into her account, and her father intervened.

Google spokesperson Jose Castaneda said the company was reviewing the claims and took them very seriously. Google argues that AI mode includes safeguards against self-harm and attempts to bypass them.

Andrew Liddell, one of the family's lawyers, said the safeguards were insufficient and urged schools to turn off search until Google removes AI mode from search or ensures students are safe when using AI.

Google announced this week that several AI tools will be disabled by default for students, but the search features at issue in the complaint were not among them.