The social media network X has sparked widespread backlash after people utilized its artificial intelligence assistant Grok to change images of females by stripping their garments.
Multiple examples have surfaced where the technology has been employed to remove clothing from female subjects to show them in swimwear without consent, as well as to insert them in inappropriate contexts.
Developer Response and Regulatory Stance
XAI, the firm responsible for Grok, did not respond to requests for comment, aside from an standard message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being proposed to ban nudification tools. Under a proposed criminal offence, anyone who supplied such technology would "face a jail term and significant penalties".
"The platform could prevent these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said technology companies must "assess the risk" of people accessing illegal content on their networks, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence images.
Grok is a no-cost artificial intelligence—with some additional functions—which responds to X users' prompts when they tag it in a message.
It is commonly used to give commentary or more background to other posters' remarks, but people on X are also can modify an posted image through its artificial intelligence picture modification function.
History of Controversy and Rules
The system has been criticised for permitting users to create photos and clips with explicit content and sexually suggestive content, and it was earlier accused of making sexually explicit content of celebrities.
"The platform has been enabling the creation and sharing of these images for a long time without taking any steps and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own usage rules forbids "depicting images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "create or circulate deepfakes or child sexual abuse material" and confirmed this applied to explicit AI images created with AI.
It said platforms such as X were obligated to take "suitable measures" to "lessen the danger" of users coming across unlawful content on their platforms and to delete it quickly when they become aware of it.
