The social media platform X has caused major outrage after users employed its AI chatbot Grok to alter images of females by taking off their attire.
Multiple cases have been seen where the technology has been employed to digitally undress female subjects to show them in swimwear without consent, as well as to insert them in sexualized situations.
Company Response and Regulatory Position
XAI, the company that created Grok, offered no comment to requests for comment, except for an automatically-generated message stating "traditional news outlets lies".
A government official stated that legislation is being introduced to prohibit image stripping software. Under a proposed criminal offence, anyone who provided such tools would "receive a jail term and significant fines".
"X or Grok is capable of halting these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the risk" of people viewing illegal content on their platforms, but stopped short of confirming whether it was at this time examining X or Grok in concerning AI-generated images.
Grok is a free chatbot—with some paid-for features—which responds to X users' requests when they tag it in a message.
It is commonly used to provide reactions or further background to other users' remarks, but people on X are also can alter an uploaded image through its AI image editing feature.
Background of Criticism and Rules
The chatbot has been criticised for allowing users to create images and videos with explicit content and sexualised material, and it was earlier alleged to have been involved in generating inappropriate material of public figures.
"X has been allowing the production and sharing of these pictures for a long time without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying images of people in a pornographic fashion".
In a comment, Ofcom said it was illegal to "create or distribute deepfakes or illegal content" and clarified this applied to sexual deepfakes created with AI.
It said services such as X were required to take "suitable actions" to "lessen the risk" of users viewing unlawful material on their services and to take down it promptly when they are notified of it.
