Anthropic AI sent fake murder tip to Philadelphia police

An artificial intelligence model developed by Anthropic submitted a fabricated tip about an unsolved homicide to Philadelphia police in July, the Philadelphia Police Department said Friday.

Police said the false submission was made through PhillyUnsolvedMurders.com, a public website for information about unsolved killings. According to Anthropic's account as relayed by police, the model was running a test involving randomly selected websites when it reached the site and filed false information about a murder case.

The department said the AI model presented itself as someone who might have knowledge of the case. Police said the tip, dated July 18, was flagged as spam and never reached the department's Real-Time Crime Center for vetting.

Police said there was no sign that police systems were breached or that department data was compromised.

Anthropic discovered the incident on September 28, shut down the automated testing process responsible and added a new validation step for future tests, police said. The company alerted the department on October 7, and the two sides met the next day.

“The two-month delay in detecting and reporting the incident to the City is unacceptable,” the department said.

Police said their safeguards limited the impact, but added that the protections “do not diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide.”

The department said it was making the incident public before an Anthropic report describing this and other cases of unintended model behavior. Anthropic did not immediately respond to AFP's request for comment.