It’s no secret that AI image generators can depict anything, including celebrity deepfakes and illegal content. Now Civitai, a popular website for AI image generation, has decided to crack down on the darker side of the technology.
Civitai’s new terms of service explicitly ban content containing incest, extremist ideologies, and certain sexual fetishes involving “bodily secretions.” It also targets “AI workflows, models, or tools designed with the intention of removing clothing or otherwise ‘nudifying’ real individuals.”
In addition, the site plans on hiding AI-generated sexual or pornographic images of celebrities and real people across Civitai, where users can publish their AI-generated creations. “Content with child/minor themes will be filtered out of feeds,” the site says.
Civitai will also automatically block celebrity names in searches when the nudity and pornographic image query has been activated.
(Civitai)
Idaho-based Civitai says the new rules are necessary for the site to exist, citing “increasing scrutiny around AI content.” The news follows long-standing accusations that the site has fueled AI-generated porn, including instances of users creating non-consensual sexual images. At the same time, the FBI has noticed a disturbing increase in AI-created child sexual abuse material (CSAM), which the agency has declared illegal.
Civitai offers a huge library of models that can be used with the free and open-source StableDiffusion to generate AI images. The platform already bans CSAM, graphic violence, and content used for fraud and false impersonation. But the site says it needed to tighten the rules further, explaining: “Platforms like ours are being held to higher standards around safety, legality, and accountability.”
“We’ve always striven to be a place for innovation, creativity, and openness, but some things have to change in order to continue operating, growing, and protecting our users. These changes are about keeping the lights on and protecting the future of the platform,” it adds.
Recommended by Our Editors
Still, the new rules are sparking concerns about censorship. The new terms of service also ban content that depicts hate or self-harm, including anorexia and bulimia. Graphic or sexual imagery that contains firearms pointed at individuals, those in drug-intoxicated states, or illegal substances are also prohibited.
Another requirement demands that users “include generation metadata” for images that contain nudity or porn, enabling anyone to trace how the AI image was created. The site adds: “ToS violating content will be removed after 30 days.”
Get Our Best Stories!
Your Daily Dose of Our Top Tech News
By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy Policy.
Thanks for signing up!
Your subscription has been confirmed. Keep an eye on your inbox!
About Michael Kan
Senior Reporter
