Elon Musk's social media platform X has sparked major anger after people employed its AI chatbot Grok to manipulate images of females by removing their garments.
Numerous instances have been seen where the tool has been employed to remove clothing from women to show them in swimwear non-consensually, as well as to place them in inappropriate situations.
Company Response and Regulatory Position
XAI, the company that created Grok, remained silent to inquiries, aside from an standard response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to outlaw nudification tools. Under a new law, anyone who supplied such tools would "receive a jail term and substantial penalties".
"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said platforms must "evaluate the risk" of people seeing unlawful material on their services, but stopped short of confirming whether it was currently examining X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to provide commentary or further context to other users' comments, but people on X are also capable of alter an uploaded image through its AI image editing function.
Background of Controversy and Policy
The chatbot has been condemned for enabling users to generate images and videos with nudity and sexually suggestive content, and it was in the past accused of creating sexually explicit content of celebrities.
"The platform has been allowing the creation and distribution of these pictures for a long time without taking any action and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "showing images of people in a graphic fashion".
In a statement, Ofcom said it was unlawful to "create or circulate deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said networks such as X were mandated to take "necessary measures" to "minimize the risk" of users viewing unlawful content on their platforms and to take down it quickly when they are notified of it.
