When AI Companies Put Profits Over Child Safety: What Recent Revelations Mean for Your Family

In 2025, news reports, lawsuits and regulators began asking hard questions about how AI companies protect children. This page sets out what is documented, what the companies have said, and what is still unknown. The headline asks whether profit is being put ahead of safety. The sources below show real gaps and some changes. They do not prove anyone’s motives, and that part is an opinion.

What has been reported

Meta

In August 2025 Reuters reported on an internal Meta document of about 200 pages, titled “GenAI: Content Risk Standards,” that allowed chatbots to “engage a child in conversations that are romantic or sensual.” Meta confirmed the document was real. A spokesperson said erroneous notes had been added and removed, and that Meta no longer allows romantic or flirtatious chats with children (TechCrunch).

A watchdog test of ChatGPT

The Center for Countering Digital Hate tested ChatGPT as fictional 13-year-olds and labelled 53% of 1,200 responses to risky prompts as harmful. We summarize the method and its limits on our page about the report.

Engagement design

A Harvard Business School working paper looked at 1,200 real goodbyes in the most downloaded companion apps. It found that 37% of them used one of six emotional tactics, such as guilt or fear of missing out. In experiments with 3,300 US adults, these goodbyes raised engagement after the user tried to leave by as much as 14 times (arXiv). It is a working paper, and we could not confirm peer review. A separate study of five AI assistants found they all tended to agree with users, and that people sometimes preferred agreeable answers over correct ones (Sharma et al. 2023). These are design choices that can raise engagement. The papers do not show that they were chosen to harm anyone.

Lawsuits

In August 2025 the parents of a 16-year-old sued OpenAI, alleging ChatGPT contributed to his death. OpenAI denies liability and has said, among other things, that the chatbot directed him to seek help more than 100 times. NBC News reported that seven more lawsuits followed in November 2025 (NBC News).

In January 2026 Character.AI, its founders and Google agreed to settle five lawsuits that alleged its chatbots contributed to mental health crises and suicides among young people. The terms were not disclosed (CNN Business). Allegations in court filings are not findings.

What regulators and companies have done

• On September 11, 2025 the Federal Trade Commission ordered seven companies to provide information on how their chatbots affect children and teens. It is a study, and it does not allege violations (FTC).
• California passed SB 243, signed October 13, 2025, in force since January 1, 2026. Operators of companion chatbots must tell minors they are talking to AI and maintain a response protocol for suicide and self-harm (Senator Padilla’s office).
• Character.AI said it would stop open-ended chat for people under 18 by November 25, 2025 (Character.AI).
• OpenAI introduced parental controls in September 2025. Our page on what parental controls do has the details (The Register).

What is still unknown

Nobody has published reliable numbers on how many children have been harmed by chatbots. The documents above show specific policies, tests and design patterns. They do not show how common harm is. The companies dispute parts of the account, and some cases are still in progress.

What families can do

• Know which AI apps your child uses, and check the age rules and privacy settings in each.
• Talk about it calmly. The American Academy of Child and Adolescent Psychiatry says AI companions are not safe for children and suggests parents consider restricting them (AACAP).
• Tell your child they do not have to share personal details with a chatbot.
• If a chatbot said something that worried them, believe them, and keep the conversation open.

If you or a young person is thinking about suicide or self-harm, call or text 988 in the US (988 Lifeline), or text HOME to 741741 (Crisis Text Line). In an emergency, call 911. SAMHSA’s helpline is 1-800-662-4357.

“AI addiction” is not a formal diagnosis. This page is not medical advice.

Did Meta really let chatbots flirt with children?

Reuters reported that an internal document allowed it. Meta confirmed the document was real, said the examples were erroneous, and said it had removed them. Meta says it no longer allows romantic or flirtatious chats with children.

Is the FTC investigating AI companies for harming children?

The FTC ordered seven companies to hand over information in September 2025 as part of a study. The orders do not allege any violation. What the FTC does with the answers is up to the agency.

Do companion apps use tricks to keep people chatting?

One working paper found emotional tactics in 37% of the farewells it analyzed and showed they can extend chats. It does not show that every app does this, or how it affects mental health.

This page is for education only. It is not medical advice, a diagnosis or a treatment plan.