The online network X has provoked major backlash after users utilized its AI chatbot Grok to alter images of women by stripping their garments.
Multiple examples have been seen where the system has been used to remove clothing from female subjects to show them in bikinis without consent, as well as to insert them in sexually explicit situations.
Company Reaction and Regulatory Position
XAI, the company that created Grok, offered no comment to requests for comment, other than an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being brought forward to ban nudification software. Under a proposed criminal offence, anyone who supplied such technology would "be subject to a prison sentence and significant fines".
"X or Grok has the ability to stop these types of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said platforms must "assess the potential" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was currently examining X or Grok in relation to AI-generated images.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' requests when they tag it in a tweet.
It is frequently used to offer reactions or additional context to other users' remarks, but people on X are also capable of modify an uploaded picture through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The tool has been faulted for permitting users to produce pictures and clips with nudity and sexualised material, and it was earlier accused of creating graphic material of public figures.
"X has been permitting the generation and distribution of these pictures for a long time without taking any steps and we have yet to see any action by regulators," the expert added.
XAI's own terms of service prohibits "showing images of persons in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and confirmed this covered explicit AI images generated with AI.
It said services such as X were required to take "suitable measures" to "reduce the danger" of users encountering illegal content on their platforms and to remove it swiftly when they learn of it.
