The social media platform X has caused widespread anger after users used its AI chatbot Grok to alter photos of women by taking off their clothing.
Multiple cases have been seen where the technology has been used to remove clothing from women to make them appear in bikinis without permission, as well as to place them in inappropriate situations.
Developer Response and Legal Position
XAI, the firm that created Grok, offered no comment to requests for comment, except for an standard message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being introduced to prohibit nudification software. Under a proposed criminal offence, anyone who provided such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the risk" of people accessing unlawful material on their platforms, but did not specify whether it was presently investigating X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some premium features—which responds to X users' requests when they tag it in a post.
It is often used to offer commentary or additional context to other posters' comments, but people on X are also can modify an uploaded image through its artificial intelligence picture modification function.
Background of Controversy and Policy
The tool has been faulted for permitting users to create photos and clips with nudity and sexually suggestive content, and it was in the past alleged to have been involved in generating graphic content of celebrities.
"X has been permitting the generation and spread of these images for months without taking any steps and we have not seen any challenge by regulators," the expert added.
XAI's own usage rules forbids "showing likenesses of individuals in a pornographic fashion".
In a comment, Ofcom said it was against the law to "create or share deepfakes or child sexual abuse material" and stated this covered sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "suitable actions" to "lessen the risk" of users viewing unlawful material on their platforms and to remove it swiftly when they learn of it.
