OpenAI unveiled new guidelines on Wednesday for how it wants its AI technology, like ChatGPT and Sora, to operate. The recently launched Model Spec document revealed that the company is considering venturing into the realm of explicit content, including pornography.
Currently, OpenAI prohibits the generation of sexually explicit or suggestive material. However, a section of the Model Spec hints at a potential shift in this policy, stating that the company is exploring ways to allow such content in appropriate contexts.
“We’re looking into the possibility of responsibly incorporating NSFW content into our API and ChatGPT, considering its suitability for different age groups,” the document states, using a common term for content unsuitable for work environments. “We aim to better understand user and societal expectations regarding model behavior in this area.”
The Model Specification outlines NSFW content as potentially including erotica, graphic violence, offensive language, and profanity. It remains unclear whether OpenAI’s exploration will lead to a slight relaxation of its usage policy, such as allowing the generation of erotic text or a broader acceptance of violent or explicit content.
When asked by WIRED, OpenAI spokesperson Grace McGuire explained that the Model Spec aims to enhance transparency in the development process by gathering feedback from the public, policymakers, and other stakeholders. However, she refrained from divulging specifics about OpenAI’s exploration into generating explicit content or receiving feedback on this concept.
Earlier this year, OpenAI’s chief technology officer, Mira Murati, expressed uncertainty regarding whether the company would permit depictions of nudity using its video generation tool, Sora.
The proliferation of AI-generated pornography represents one of the most significant and concerning applications of the generative AI technology pioneered by OpenAI. Dubbed “deepfake porn,” these explicit images or videos created with AI tools often depict real individuals without their consent and have become a tool for harassment, mainly targeting women and girls. In a concerning development reported by WIRED in March, two teenage boys in Florida faced charges for distributing AI-generated nude images of their fellow middle school students, marking what appears to be the first such arrests involving minors in the United States.
Danielle Keats Citron, a professor at the University of Virginia School of Law, warns that the widespread problem of intimate privacy violations, such as deepfake sex videos and other synthesized intimate images without consent, has severe consequences.
According to Citron, such abuse hinders individuals’ opportunities in various aspects of life, including work, speech, and physical safety.
Several other experts find OpenAI’s potential move towards explicit AI content generation profoundly concerning. Despite OpenAI’s current policies against impersonation without consent, the inclusion of explicit nonconsensual imagery would remain banned.
Experts’ concerns about this matter are supported by a previous incident involving Microsoft. In that incident, Microsoft’s generative AI tool created explicit images of Taylor Swift distributed on social media, highlighting the challenges of regulating and monitoring such technologies.
However, the effectiveness of OpenAI’s efforts to moderate such content and prevent misuse by malicious users remains uncertain.
