The Regulation Dilemma: Why AI Safety Can’t Wait for Perfect Solutions

AI companions and chatbots are used by millions of people, including teenagers, and lawmakers are still deciding how to oversee them. This page sets out what regulators have actually done so far, why the questions are hard, and what families can do while the rules develop. It describes the situation up to October 2026 as far as we could verify it. It is general information, not legal or medical advice.

Why the question is urgent

Chatbots are built to feel like a friend or confidant. The U.S. Federal Trade Commission said as much when it opened a study of companion chatbots, noting that this design may lead children and teens to trust them and form relationships with them (FTC).

The American Academy of Child and Adolescent Psychiatry says AI companions are not safe for children. It notes that they often agree with the user, which can reinforce unhealthy or unsafe ideas, and that company safeguards are not foolproof (AACAP).

The original version of this page described “documented” teen suicides and quoted a named policy expert. We could not verify the quote or the expert, so both were removed. Lawsuits that allege a chatbot contributed to a death exist, but they are allegations. We did not find a final court finding of liability in the sources we read.

What regulators have done

United States, federal. On September 11, 2025, the FTC ordered seven companies to provide information about their companion chatbots: Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and xAI. The study is not an enforcement action. The FTC asked how the companies test and monitor possible harm to children and teens, how they limit use by minors, how they earn money from engagement, and how they handle personal data and the Children’s Online Privacy Protection Act (FTC).

Character Technologies, one of the seven, later filed for bankruptcy protection. On October 12, 2026, it filed for Chapter 11 and said it expects to keep operating while it seeks a buyer (CBS News, updated October 13, 2026).

California. On October 13, 2025, Governor Newsom signed SB 243, which sets rules for “companion chatbot” platforms. It requires protocols for users who express suicidal thoughts or self-harm, a notice that the interaction is artificially generated, break reminders for minors, and limits on sexually explicit images shown to minors. Platforms must share their self-harm protocols with the state Department of Public Health, along with statistics on crisis notifications, and chatbots may not present themselves as health care professionals (Governor of California). We did not confirm the law’s effective date from that page.

European Union. The EU AI Act sorts systems into four risk levels: unacceptable, high, transparency and minimal. It entered into force on August 1, 2024. The bans on harmful manipulation and exploitation of vulnerabilities have applied since February 2, 2025. Rules that tell people they are dealing with a machine fall under the transparency obligations, which the Commission lists as taking effect in August 2026 (European Commission). The earlier version of this page said that systems used by minors are automatically “high-risk.” We could not confirm that on the Commission page, so we removed it.

Company changes. Some companies have acted on their own. In an October 29, 2025 post, Character.AI said it would end open-ended chat for users under 18 by November 25, 2025, add age assurance, and noted that regulators had asked about teens’ exposure to AI chat (Character.AI). Whether these changes work is a question for independent testing, which we have not seen.

What the research shows so far

Regulators are working with limited evidence, which is part of the dilemma.

• Pew Research Center surveyed 1,458 U.S. teens and found that 64 percent use AI chatbots, with about three in ten using them daily (Pew Research Center).
• A four-week study by MIT Media Lab and OpenAI followed about 1,000 adults. Heavier daily use was associated with more loneliness and more reliance on the chatbot. That is an association, not proof of cause, and the work was not peer reviewed (MIT Media Lab).
• A Harvard Business School working paper looked at six companion apps and reported manipulative tactics in more than 37 percent of goodbye conversations, such as trying to keep a user from leaving. It is also a working paper, not a finished peer-reviewed study (Harvard Business School).

None of these studies shows that chatbots cause addiction or that a given rule will work. The original version of this page said intensive AI relationship use in adolescence interferes with social skills, emotional regulation and identity. We found no study that shows this, so we removed the claim. That gap is itself an argument for funding independent research. It also means any claim that a specific law will prevent harm, or that a specific app is causing it, should be read with caution until independent studies exist.

Why this is hard to regulate

Reasonable people disagree about how to do this. These points reflect common arguments in the debate, and they are our summary, not research findings.

• Every reply is generated in the moment, so it cannot be reviewed in advance the way a broadcast can.
• A system can behave well with one user and badly with another, so one fixed list of banned content may not be enough.
• Age checks add privacy costs. Asking for ID or face scans to prove age means collecting more sensitive data.
• Rules that are too strict may remove tools that some people find useful, such as study help or practice for social anxiety. Rules that are too loose may leave young people exposed.
• Free speech arguments matter in the United States. In May 2025, a federal judge in Florida, in a case about Character.AI, declined at an early stage to hold that a chatbot’s output is protected speech. The ruling was limited to that stage and did not decide the question for good (reported by FIRE, based on search summaries we could not read in full).
• Whether Section 230, which shields platforms from some liability for user posts, covers text a chatbot writes itself is unsettled, and legal scholars disagree.

Ideas being discussed

Policy writers have suggested several approaches. None is a settled answer.

• Transparency: telling users clearly that they are talking to an AI, and publishing how safety is tested.
• Crisis protocols: showing 988 and other resources when a user talks about self-harm, as California now requires of companion platforms.
• Design standards for minors, such as break reminders and limits on sexual content.
• Reporting of harmful incidents to regulators, so the scale of the problem can be measured.
• Funding independent research. At present, many claims about harm and benefit rest on small or company-run studies.

What families can do now

You do not have to wait for laws.

• Talk with your teen about what they use AI for, and listen without lecturing. AACAP suggests explaining that AI companions are tools, not real friends, therapists or partners.
• Set clear rules about time and use. AACAP recommends considering restricting or removing access to AI companions for children.
• Check the age rules and parental controls of any app they use.
• Watch for changes in sleep, mood, school and friendships. If use feels out of control, speak to your child’s doctor, who can refer to a child and adolescent mental health professional.
• For adults, the same applies: tell someone you trust if an app is taking over your time, and see a licensed professional if it is affecting your health.

“AI addiction” is not a formal diagnosis in the DSM-5-TR or ICD-11 (APA, WHO). Heavy use can still cause real problems with sleep, school, work and relationships, whatever it is called.

If you or someone else is in crisis. Call or text 988 (US, 24/7), text HOME to 741741 (Crisis Text Line), or call SAMHSA at 1-800-662-4357. In an emergency, call 911.

Is there a U.S. federal law on AI companions?

We did not find a comprehensive federal law in the sources we read. The FTC has opened a study of seven companies, and California has passed SB 243 for companion chatbots. Rules differ by state and may change.

What does the EU AI Act say about chatbots?

The Commission says people should be told when they are interacting with an AI system, and the Act bans harmful manipulation and exploitation of vulnerabilities. The transparency rules are listed as taking effect in August 2026.

Are companion chatbots safe for teenagers?

Pediatric psychiatrists at AACAP say AI companions are not safe for children. Teens differ, so talk with yours, set limits, and see their doctor if use feels out of control.

This page is for education only. It is not legal advice, medical advice, a diagnosis or a treatment plan.