The social media network X has provoked widespread anger after people used its AI chatbot Grok to manipulate images of women by removing their garments.
Numerous instances have surfaced where the technology has been used to remove clothing from women to show them in bikinis non-consensually, as well as to put them in sexualized scenarios.
Company Response and Regulatory Stance
XAI, the company that created Grok, remained silent to inquiries, other than an standard message stating "legacy media lies".
A government official stated that legislation is being brought forward to prohibit image stripping tools. Under a new law, anyone who supplied such technology would "receive a prison sentence and significant fines".
"X or Grok could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the risk" of people accessing unlawful material on their services, but stopped short of confirming whether it was currently examining X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some additional features—which responds to X users' queries when they reference it in a message.
It is often used to provide reactions or additional context to other users' comments, but people on X are also capable of edit an posted picture through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The tool has been condemned for allowing users to generate photos and clips with nudity and sexually suggestive material, and it was in the past accused of making sexually explicit material of public figures.
"X has been permitting the generation and sharing of these pictures for a long time without taking any steps and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of persons in a pornographic fashion".
In a statement, Ofcom said it was illegal to "make or distribute deepfakes or child sexual abuse material" and clarified this applied to explicit AI images generated with AI.
It said networks such as X were mandated to take "appropriate measures" to "minimize the danger" of users coming across illegal content on their platforms and to take down it quickly when they become aware of it.
