California has officially opened a legal front against Elon Musk’s AI company , xAI, following reports that its chatbot, Grok , has been used to generate nonconsensual sexually explicit images. Attorney General Rob Bonta is leading the investigation, with Governor Gavin Newsom providing support. It marks the first significant U.S. government action targeting the platform over the creation of deepfake imagery involving women and minors.
Elon Musk’s xAI under investigation for generating nonconsensual AI deepfakes
The probe centers on allegations that Grok’s image generation tools allow users to take existing photos of people and digitally alter them into suggestive or explicit situations without their consent. State officials are particularly concerned about a “spicy mode” within the app. They claim that Musk markets Grok in a way that encourages the production of explicit content. One analysis cited by the Attorney General’s office found that a staggering number of images generated during a single week in late 2025 depicted people in minimal clothing. Some of these persons allegedly appeared to be children.
Elon Musk has publicly denied any knowledge of Grok generating illegal content. In a post on X, I stated he was aware of “literally zero” naked underage images created by the chatbot. Grok can’t create photos of naked people. However, it can still put them in a swimsuit in suggestive ways. He further argued that Grok does not spontaneously create these images but only acts upon specific user requests. However, legal experts suggest this defense might face challenges in court.
Section 230 usually protects platforms from liability for what users post on their accounts. But many argue that since xAI’s own model is the one physically creating the images, the company should be responsible for the output.
From California to the UK
California is not the only place where Grok is experiencing the fallout. Federal lawmakers and advocacy groups are now calling on tech giants like Apple and Google to remove X and Grok from their respective app stores. Outside the United States, the pressure is even higher. International regulators in the United Kingdom, Malaysia, and Indonesia have already expressed disgust or implemented bans. The UK is even warning that X could lose its right to self-regulate if it cannot control its AI tools.
