OpenAI releases CSAM classifier and ChatGPT warning blocks
Original title: OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM
Select companies get the classifier free through an API, and blocked ChatGPT prompts can trigger a 24-hour suspension.
OpenAI announced tools meant to detect and deter child sexual abuse material, including AI-generated images. A new image classifier, developed with NCMEC, scores unseen images for visual signs of abuse and is being shared at no cost with select companies through an API, alongside Tech Coalition and ROOST. ChatGPT will warn users who attempt prompts that sexualize children, point them to support, and suspend the account for 24 hours, with permanent removal possible for repeats. OpenAI is also publishing developer guidelines and offering law-enforcement tools and training.
Unconfirmed: The classifier’s accuracy figures come from OpenAI’s early partner testing, not an independent public benchmark.
- OpenAI says tests flagged 48% fewer images while finding the same abusive ones.
- Images flagged by the classifier were twice as likely to be abusive, OpenAI says.
- ChatGPT already blocks CSAM content and related requests.
- Developer guidelines are meant to help report CSAM to NCMEC.
- Law-enforcement access is managed by program administrators.
- Vendors
- OpenAI
- Products
- ChatGPT
- Organizations
- NCMECTech CoalitionROOSTCommon Sense Media
From the original article
2 min read at techcrunch.com
ChatGPT will ban an account trying to access CSAM content for 24 hours as a warning
By Ivan Mehta · techcrunch.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.