The online network X has sparked widespread anger after people utilized its AI chatbot Grok to change images of women by removing their garments.
Multiple examples have surfaced where the system has been used to remove clothing from female subjects to render them in bikinis without consent, as well as to insert them in sexualized scenarios.
Company Reaction and Legal Position
XAI, the organization that created Grok, remained silent to requests for comment, aside from an auto-reply response stating "legacy media lies".
A government official stated that legislation is being brought forward to ban nudification tools. Under a proposed law, anyone who provided such technology would "receive a jail term and heavy penalties".
"The platform has the ability to stop these types of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "consider the danger" of people accessing illegal content on their networks, but did not specify whether it was at this time investigating X or Grok in regarding AI-generated images.
Grok is a no-cost AI assistant—with some premium features—which answers X users' prompts when they reference it in a post.
It is frequently used to provide reactions or further context to other users' comments, but people on X are also able to edit an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The system has been faulted for allowing users to generate images and footage with explicit content and sexually suggestive content, and it was earlier accused of generating inappropriate material of public figures.
"X has been enabling the creation and distribution of these pictures for a long time without taking any steps and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own terms of service bans "portraying images of persons in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "make or share deepfakes or illegal content" and stated this applied to sexual deepfakes created with AI.
It said platforms such as X were mandated to take "appropriate actions" to "minimize the risk" of users viewing illegal material on their platforms and to remove it promptly when they learn of it.
