The social media network X has provoked significant backlash after people employed its AI chatbot Grok to manipulate photos of women by removing their garments.
Multiple instances have surfaced where the technology has been employed to virtually strip women to render them in swimwear without permission, as well as to put them in sexually explicit situations.
Company Response and Regulatory Stance
XAI, the organization behind Grok, did not respond to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to ban nudification tools. Under a proposed criminal offence, anyone who distributed such tools would "face a prison sentence and significant penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said technology companies must "evaluate the potential" of people seeing unlawful material on their networks, but did not specify whether it was currently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some premium features—which answers X users' prompts when they reference it in a tweet.
It is commonly used to provide commentary or additional background to other posters' comments, but people on X are also able to edit an uploaded image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been criticised for permitting users to generate images and videos with explicit content and sexualised material, and it was previously alleged to have been involved in creating inappropriate content of celebrities.
"The platform has been permitting the generation and distribution of these pictures for months without taking any steps and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying likenesses of persons in a pornographic way".
In a comment, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or illegal content" and stated this covered sexual deepfakes created with AI.
It said services such as X were obligated to take "appropriate measures" to "reduce the risk" of users coming across illegal content on their services and to take down it quickly when they are notified of it.
