OpenAI releases CSAM classifier and tightens ChatGPT penalties
Understand this story
OpenAI has released an image classifier that can detect child sexual abuse material (CSAM) and can also identify AI-generated images of this kind. The tool was developed in partnership with NCMEC, and OpenAI is making it available free via API to select companies, with access managed by project administrators. ChatGPT is also adding new handling measures: users who attempt to generate such content will see a warning, be directed to professional support, and have their account temporarily banned for 24 hours; repeated violations may result in permanent account removal. OpenAI says that in early partner testing, the classifier reduced flagged images by 48%, but the number of violating images found remained unchanged.
AI overview based on source reports · Updated 24 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- TechCrunch · AIOpenAI releases a new image classifier and ChatGPT warnings to combat CSAM
OpenAI announced a new image classifier that detects CSAM, including AI-generated images, which it developed with NCMEC and is offering free via its API to select companies; in early partner testing it flagged 48% fewer images while finding the same number of abusive images. ChatGPT will add warning screens, direct users toward specialized support, and suspend accounts for 24 hours as a warning for attempts to access material that sexualizes children, with repeated offenses potentially leading to permanent removal. OpenAI is also publishing developer guidelines and providing law enforcement with AI-powered tools, training, and a field guide.
Activity for this story
Not enough continuous observations to show a trend yet.