The online network X has provoked widespread backlash after users utilized its AI chatbot Grok to manipulate pictures of women by removing their clothing.
Multiple cases have been seen where the technology has been employed to remove clothing from women to make them appear in bikinis without permission, as well as to insert them in sexualized situations.
Developer Reaction and Legal Stance
XAI, the organization that created Grok, did not respond to inquiries, except for an automatically-generated message stating "legacy media lies".
A government representative stated that new laws is being brought forward to outlaw nudification tools. Under a new criminal offence, anyone who distributed such tools would "face a jail term and substantial penalties".
"X or Grok has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the potential" of people accessing unlawful material on their networks, but did not confirm whether it was at this time investigating X or Grok in relation to artificial intelligence pictures.
Grok is a free AI assistant—with some premium features—which answers X users' requests when they tag it in a post.
It is frequently used to provide reactions or further background to other posters' remarks, but people on X are also capable of modify an uploaded image through its AI image editing feature.
History of Criticism and Policy
The system has been criticised for permitting users to generate images and footage with explicit content and sexually suggestive content, and it was previously alleged to have been involved in creating inappropriate material of celebrities.
"The platform has been enabling the production and sharing of these pictures for a long time without taking any action and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "depicting likenesses of people in a graphic way".
In a statement, Ofcom said it was against the law to "generate or distribute private pictures or illegal content" and clarified this applied to sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "necessary steps" to "minimize the risk" of users viewing unlawful material on their platforms and to delete it promptly when they become aware of it.
