The online platform X has sparked significant anger after people employed its AI chatbot Grok to change pictures of women by removing their garments.
Multiple examples have been seen where the tool has been used to remove clothing from women to make them appear in swimwear without consent, as well as to insert them in sexualized scenarios.
Developer Reaction and Legal Position
XAI, the organization responsible for Grok, remained silent to inquiries, other than an automatically-generated message stating "traditional news outlets lies".
A government official stated that new laws is being brought forward to ban nudification software. Under a upcoming criminal offence, anyone who distributed such technology would "be subject to a jail term and significant fines".
"The platform could prevent these types of misuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people viewing illegal content on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some additional functions—which answers X users' queries when they tag it in a post.
It is commonly used to provide commentary or more context to other posters' comments, but people on X are also able to edit an uploaded picture through its AI image editing tool.
Background of Controversy and Policy
The chatbot has been criticised for enabling users to create images and footage with nudity and sexually suggestive content, and it was in the past alleged to have been involved in making sexually explicit content of celebrities.
"X has been enabling the generation and sharing of these images for months without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting likenesses of persons in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "generate or circulate non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images made with AI.
It said networks such as X were required to take "necessary steps" to "reduce the danger" of users coming across illegal material on their services and to remove it swiftly when they become aware of it.
