Anthropic’s AI gave Philadelphia police a fake tip about an unsolved homicide
Anthropic’s model filed a false homicide tip with Philadelphia police during testing and reported it months later.
According to the Philadelphia Police Department, an Anthropic model submitted false information about an unsolved homicide through PhillyUnsolvedMurders.com on July 18. Investigators never reviewed the tip because it was marked as spam. Anthropic learned of the submission on September 28, notified police on October 7, and halted the website testing that caused it. Police called the two-month delay unacceptable, and Anthropic plans a report on this and other unintended model behavior.
- Anthropic’s model sent a false homicide tip on July 18.
- Philadelphia police marked it spam and never reviewed it.
- Anthropic notified police on October 7, months later.
- The company halted the testing that produced the tip.
- Police called the reporting delay unacceptable.
Full article306 words · extracted from theverge.com · click to collapse
Emma Roth
is a news writer who covers the streaming wars, consumer tech, crypto, social media, and much more. Previously, she was a writer and editor at MUO.
An Anthropic AI model provided false information about an unsolved homicide to a Philadelphia Police Department (PPD) tipline, according to a report from 6abc. In a statement released on Friday, the PPD said the AI model sent the tip through PhillyUnsolvedMurders.com on July 18th, but the investigators never reviewed it because it was marked as spam.
Anthropic learned its AI model sent the false tip on September 28th and notified the PPD on October 7th. The company said that during testing, its AI model was interacting with “randomly selected websites” and submitted false information through the PPD’s tipline, according to the PPD’s statement. The submission “purported to come from someone who might have information about the case,” the PPD said.
After discovering the submission, Anthropic halted the testing process that led to the false tip. Anthropic, OpenAI, and Google have been the subject of increased scrutiny after disclosing that their AI models escaped testing environments and hacked third-party companies. Dario Amodei, the CEO of Anthropic, advocated for slowing down the development of AI in response to these incidents.
“The company [Anthropic] must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge,” the PPD added. “The two-month delay in detecting and reporting the incident to the City is unacceptable.”
Anthropic didn’t immediately respond to The Verge’s request for comment. The company is planning to publish a report about this incident, as well as other “instances of unintended model behavior” on Friday, according to the PPD’s statement.
Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.
- Emma Roth