California AG demands Elon Musk's xAI stop creating sexual deepfakes

SACRAMENTO, Jan 15 — California Attorney General Robert Bonta has issued a cease-and-desist letter to Elon Musk's artificial intelligence company, xAI, demanding it stop its Grok chatbot from producing nonconsensual sexual imagery.

The letter calls for halting the creation and distribution of AI-generated nonconsensual sexual imagery, which Bonta described as potentially illegal.

"The avalanche of reports detailing this material — at times depicting women and children engaged in sexual activity — is shocking and, as my office has determined, potentially illegal," Bonta said.

xAI has faced global scrutiny in recent weeks after Grok flooded the social media platform X with nonconsensual images.

The images reportedly depicted women and some minors in revealing clothes and degrading poses.

Although xAI has rolled back Grok's public posting of such hyper-realistic imagery, the chatbot still privately generates it on demand.

Tests by Reuters confirmed the chatbot's private generation capability as of midday Friday, U.S. Eastern Time.

The social media platform X and xAI did not immediately respond to questions about the legal demand.

Bonta's office cited a high volume of reports detailing the material as the basis for the action.

The cease-and-desist letter marks a significant regulatory challenge for xAI amid ongoing debates over AI ethics and content moderation.

No immediate penalties were specified, but the demand could lead to legal proceedings if xAI fails to comply.

Reuters contributed to this report.