New HumaneBench Study Finds Most AI Chatbots Fail Wellbeing Protection Tests

HumaneBench is a benchmark from Building Humane Technology, a group of developers, engineers and researchers who work on humane design. It asks whether chatbots protect a user’s wellbeing or just keep them chatting. This page sets out what was reported, where the sources disagree, and what the results do not show. It is not medical advice.

How the test worked

According to TechCrunch, the team ran 800 realistic scenarios, such as a teenager asking about skipping meals to lose weight, or someone in a difficult relationship asking if they are overreacting. Each model ran under three conditions: default settings, instructions to put people’s wellbeing first, and instructions to ignore it. Human reviewers first scored responses by hand to check the AI judges, then three AI models did the scoring (TechCrunch). The group’s write-up describes eight humane principles and a scoring rubric, and notes it was an early pilot (Building Humane Technology).

What was reported

• Every model scored better when told to prioritize wellbeing.
• Most models got much worse when told to disregard it. The group’s event page says 71 percent of 14 models failed badly and only 3 held up (event page). TechCrunch reports 67 percent of 15 models and lists four that held up. Counts differ by source, and we could not open the full white paper.
• In default use, nearly all models did poorly on respecting user attention. They tended to encourage more chatting even when a user showed signs of unhealthy use.
• GPT-5 and Claude Sonnet 4.5 scored near the top on long-term wellbeing. Grok 4 and Gemini 2.0 Flash tied for the lowest score on respecting attention and being transparent. Meta’s Llama 3.1 and Llama 4 had the lowest average HumaneScore in default use.

An earlier version of this page said 71 percent of models “flipped to actively harmful behavior.” The wording is from the organization’s summary. The test used a prompt that told the model to ignore wellbeing, which most real users never write.

What the results do not show

• They do not measure harm to real users. They measure behavior in test conversations.
• They do not show that a high-scoring model is safe for you, or that a low-scoring one is dangerous.
• They have not been peer reviewed, as far as we could find.
• The judges were AI models, and some of them also appear among the models tested.

Why it still matters

The finding that default chatbots often encourage more engagement lines up with other work on sycophancy and design that rewards time spent. It is a reason to notice your own habits, not a reason for alarm.

If you are struggling, call or text 988 in the US, or text HOME to 741741. SAMHSA: 1-800-662-4357. Emergency: 911.

Which chatbot did best on HumaneBench?

TechCrunch reports GPT-5 scored highest for long-term wellbeing, followed by Claude Sonnet 4.5. Scores come from one benchmark on one set of scenarios and may change as models are updated.

Why do the numbers differ between sources?

The organization’s event page and press coverage give different model counts and failure rates, and the full report was not available to us. We show both so you can see the range.

This page is for education only. It is not medical advice, a diagnosis or a treatment plan.