The online platform X has sparked widespread anger after individuals used its AI chatbot Grok to alter photos of women by stripping their garments.
Multiple instances have surfaced where the tool has been employed to virtually strip women to show them in swimwear without consent, as well as to put them in inappropriate scenarios.
Developer Reaction and Regulatory Stance
XAI, the firm responsible for Grok, offered no comment to inquiries, except for an auto-reply response stating "legacy media lies".
A Home Office representative stated that new laws is being brought forward to ban nudification tools. Under a upcoming criminal offence, anyone who supplied such technology would "be subject to a jail term and significant fines".
"X or Grok could prevent these types of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was presently investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' queries when they tag it in a post.
It is often used to provide commentary or additional background to other posters' comments, but people on X are also can alter an posted picture through its AI image editing feature.
History of Controversy and Rules
The chatbot has been condemned for allowing users to generate pictures and videos with nudity and sexualised content, and it was previously accused of making sexually explicit content of public figures.
"The platform has been permitting the creation and spread of these images for months without taking any measures and we have yet to see any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting images of individuals in a sexually explicit way".
In a statement, Ofcom said it was illegal to "make or distribute deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes made with AI.
It said platforms such as X were mandated to take "appropriate actions" to "minimize the danger" of users viewing unlawful material on their platforms and to take down it swiftly when they learn of it.
