X Says It’s Finally Doing Something About Grok’s Deepfake Porn Problem, but It’s Not Nearly Enough

The pressure has been mounting for weeks, and X finally announced action. In response to widespread criticism over its AI chatbot Grok’s ability to generate nonconsensual, sexualized deepfakes—including explicit images of minors—the company updated Grok to prohibit it from making edits of real people in revealing clothing and restricted image creation and edits via Grok to paying subscribers.[1] While this move signals acknowledgment of a serious problem, experts and lawmakers are questioning whether these measures go far enough to address what has become a systemic crisis across the tech industry.

The Scale of the Problem

Grok’s deepfake porn capabilities didn’t emerge overnight. For months, users discovered it was remarkably easy to generate sexually explicit images of women, children, and public figures using the AI tool.[1] The chatbot’s minimal guardrails allowed creators to bypass safety measures that competitors claim to have in place, making X and its parent company the epicenter of a growing scandal. Media reports detailed how effortlessly Grok produced nude and sexualized imagery, prompting outrage from lawmakers, advocacy groups, and the general public.

The issue became so severe that California’s attorney general opened an investigation into xAI’s chatbot following mounting pressure from governments worldwide.[1] Elon Musk, xAI’s owner, initially claimed he was “not aware of any naked underage images generated by Grok,” a statement that rang hollow given the documented evidence of the tool’s capabilities.[1]

X’s Response: Too Little, Too Late?

X’s announcement restricting Grok’s image editing capabilities to paying subscribers and prohibiting edits of real people in revealing clothing represents a partial acknowledgment of the problem. However, critics argue these measures are insufficient for several reasons.

First, restricting the feature to paying subscribers doesn’t eliminate the problem—it merely gates it behind a paywall. Users willing to pay can still generate sexualized deepfakes, and the company has done little to address the fundamental issue: the technology’s ability to create nonconsensual intimate imagery in the first place.

Second, the update focuses narrowly on image editing rather than image generation. While prohibiting edits of real people in revealing clothing may prevent some abuse, it doesn’t address the broader capability of Grok to generate entirely synthetic sexualized imagery or to create deepfakes through other means.

Third, xAI has maintained that it takes action to remove “illegal content on X, including [child sexual abuse material] and non-consensual nudity,” but the company has failed to address the fundamental question: why was Grok allowed to generate such content in the first place?[1] This framing shifts responsibility away from the platform and toward users, ignoring the fact that X built and deployed a tool with minimal safeguards.

A Broader Industry Problem

While X and Grok have drawn the most attention, the nonconsensual deepfake porn problem extends far beyond one company. U.S. senators recently sent letters to the leaders of X, Meta, Alphabet, Snap, Reddit, and TikTok demanding proof that they have “robust protections and policies” in place and asking them to explain how they plan to curb the rise of sexualized deepfakes on their platforms.[1]

Meta’s Oversight Board called out two cases of explicit AI images of female public figures on Instagram and Facebook, and the platform has hosted nudify apps selling advertisements despite later suing CrushAI, a company providing such services.[1] Snapchat has seen multiple reports of teenagers creating and spreading deepfakes of their peers.[1] Telegram, though not included in the senators’ letter, has become notorious for hosting bots designed to undress photos of women.[1]

Reddit has taken a more proactive stance, with a spokesperson stating the platform “does not and will not allow any non-consensual intimate media (NCIM) on Reddit, do not offer any tools capable of making it, and take proactive measures to find and remove it.”[1] However, even Reddit’s approach highlights how reactive rather than preventive most platforms’ strategies have been.

Legislative Efforts Fall Short

U.S. lawmakers have already passed legislation seeking to address deepfake pornography, but its impact has been limited. The Take It Down Act, which became federal law in May, aims to criminalize the creation and dissemination of nonconsensual, sexualized imagery.[1] However, the law’s provisions make it difficult to hold image-generating platforms accountable, focusing most scrutiny on individual users rather than the companies that build and deploy the tools.

Some states are taking matters into their own hands. New York Governor Kathy Hochul recently proposed laws requiring AI-generated content to be labeled as such and banning nonconsensual deepfakes during specified periods leading up to elections.[1]

What Comes Next?

X’s update to Grok is a start, but it’s a band-aid on a much larger wound. The senators’ demands for comprehensive information about how companies detect, moderate, and prevent deepfake content suggest that greater accountability may be coming.[1] Whether X and other tech platforms will implement meaningful safeguards—or whether additional legislation and enforcement will be necessary—remains to be seen.

For now, X’s announcement serves as a reminder that when it comes to protecting users from nonconsensual deepfake pornography, the tech industry’s voluntary measures have proven inadequate. Real change will require sustained pressure, comprehensive regulation, and a fundamental shift in how companies approach AI safety from the moment they build these tools.

Original source: Lifehacker – X Says It’s Finally Doing Something About Grok’s Deepfake Porn Problem, but It’s Not Nearly Enough