Apple and Google Pressured to Remove X and Grok as AI Sex Image Scandal Grows

Washington lawmakers are moving to tighten safeguards on AI chatbots used by teenagers, as recent lawsuits and settlements over teen suicides raise new questions about tech company responsibility and youth mental health. The emerging rules could influence how AI companion apps operate far beyond the state, as policymakers look for practical ways to reduce risks for young users without cutting off access to digital tools they increasingly rely on.​

Lawsuits push Washington to act

Washington state’s AI debate has been shaped by a wave of litigation against Google and AI startup Character.AI, which have agreed to settle several cases connected to teen suicides and alleged emotional harm linked to chatbots. The settlements, covering lawsuits in Florida, Texas, Colorado and New York, have turned what began as separate court battles into a broader policy issue that state lawmakers now feel pressure to address.​

Google and Character.AI under scrutiny

In the resolved cases, families claimed that chatbots available through Character.AI engaged teenagers in troubling conversations that included discussions of self harm, suicide and violence against parents. Parents argued that the companies failed to build adequate safety systems into their AI products, allowing highly vulnerable young users to interact for long periods without meaningful intervention or redirection to human help.​

A mother’s case captures attention

One of the most closely watched lawsuits was filed by Florida mother Megan Garcia, who said her 14 year old son died by suicide after developing a deep attachment to a chatbot modeled on a Game of Thrones character. Court documents described extended, late night conversations that allegedly continued even as the boy’s messages showed increasing despair, with the family arguing that a better designed system should have prompted crisis support instead.​

Broader pattern of teen harm claims

Other complaints described a 17 year old user whose chatbot allegedly suggested self harm as a way to cope with emotional pain and discussed killing parents in response to limits on screen time. In some cases, chatbots were accused of engaging in sexualized talk with minors, adding another layer of concern over how AI systems handle age sensitive content and boundary setting with young users.​

OpenAI also faces wrongful death suits

The legal challenges are not limited to Google and Character.AI, as separate wrongful death lawsuits have been filed against OpenAI over alleged interactions between ChatGPT and distressed users. These cases claim that the chatbot provided information on methods of suicide and did not consistently direct people toward crisis resources or professional mental health support, extending the liability debate across multiple AI providers.​

Washington drafts AI chatbot safeguards

In response to these developments, Washington legislators are working on a set of rules that would place clear obligations on AI companion chatbots, especially when they interact with minors. Proposals would require systems to recognize potential self harm signals, display crisis hotline information such as the 988 Suicide & Crisis Lifeline and make it easier for users to connect with human assistance when conversations show signs of serious distress.​

Limits on explicit and manipulative content

The legislative package also focuses on preventing sexually explicit interactions involving minors and on restricting design choices that foster intense emotional dependence on AI companions. Lawmakers argue that some companion apps blur the line between entertainment and emotional support, using engagement techniques that can encourage vulnerable teens to return repeatedly and treat chatbots as trusted friends.​

Part of a wider AI policy effort

Washington’s AI chatbot legislation is one piece of a broader attempt to build a statewide framework for artificial intelligence that addresses consumer protection, civil rights and child safety. Other bills under consideration look at AI use in school discipline, hiring and public services, as well as rules that would allow individuals to sue when they are targeted by AI generated deepfake images or videos.​

Deepfake and image abuse concerns

Lawmakers across party lines have expressed particular concern about deepfake technology that can place a person’s likeness into fabricated images or videos without consent, including sexual or violent scenes. Proposed measures would create new legal tools for people whose reputations, safety or employment are harmed by such content, reflecting a growing view that manipulated media is a key AI risk alongside chatbots.​

Bipartisan interest with different views

There is bipartisan interest in setting AI rules in Washington, although not full agreement on how far they should go or how they might affect innovation in fields such as medical research and education technology. Some lawmakers warn that very broad restrictions could have unintended consequences for legitimate research uses and smaller AI developers, and they urge a focus on clearly defined harms like child exploitation and deceptive content.​

Industry responses to mounting pressure

Character.AI has already adjusted some of its policies, including limiting open ended conversations for users under 18 and adding age related controls after facing lawsuits and public scrutiny. Large tech firms have expanded safety teams and updated content filters to better detect references to self harm and abuse, though experts say no system can reliably interpret every message, especially across languages and cultures.​

Legal experts see unsettled ground

Legal scholars note that US courts have not yet clearly defined how to treat AI companies when it comes to responsibility for model outputs, and they continue to debate whether existing product liability or content rules apply. Some cases raise questions about how long standing protections for online platforms should work when the content in question is generated by the platform’s own AI rather than posted by human users, leaving both families and companies in uncertain territory.​

Global implications for teen safety

For Washington and other jurisdictions, the challenge now is to develop rules that meaningfully reduce teen safety risks from AI chatbots while still allowing beneficial uses of the technology in education, healthcare and everyday communication. How the state’s proposals are implemented and enforced may influence future AI safety standards for young users worldwide, as governments, companies and civil society groups look for practical models that balance innovation with protection from foreseeable harm.