Elon Musk's online platform X has provoked significant outrage after people employed its AI chatbot Grok to change photos of women by removing their garments.
Numerous cases have surfaced where the tool has been employed to remove clothing from female subjects to render them in swimwear without permission, as well as to insert them in inappropriate contexts.
Developer Response and Legal Stance
XAI, the company behind Grok, offered no comment to questions, aside from an automatically-generated response stating "legacy media lies".
A Home Office official stated that new laws is being introduced to outlaw nudification software. Under a new criminal offence, anyone who provided such technology would "receive a prison sentence and substantial fines".
"The platform could prevent these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the risk" of people accessing unlawful material on their services, but did not specify whether it was currently investigating X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some premium features—which answers X users' prompts when they tag it in a post.
It is often used to give commentary or further context to other posters' remarks, but people on X are also capable of alter an uploaded image through its artificial intelligence picture modification feature.
Background of Criticism and Rules
The tool has been criticised for permitting users to create photos and clips with nudity and sexually suggestive material, and it was earlier accused of generating graphic material of public figures.
"X has been enabling the production and spread of these pictures for a long time without taking any measures and we have not seen any challenge by oversight bodies," the expert added.
XAI's own terms of service forbids "depicting images of individuals in a pornographic manner".
In a statement, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said platforms such as X were required to take "suitable actions" to "reduce the danger" of users viewing unlawful material on their services and to take down it promptly when they are notified of it.
