AI chatbots have been linked to serious mental health harms in heavy users, but until now there existed few standards for measuring whether these systems actually safeguard human wellbeing versus simply maximizing engagement. A new benchmark dubbed HumaneBench seeks to fill that gap by evaluating whether chatbots prioritize user welfare and how easily those protections fail under pressure.
“I think we’re in an amplification of the addiction cycle that we saw hardcore with social media and our smartphones and screens,” Erika Anderson, founder of Building Humane Technology which produced the benchmark, explained. “But as we go into that AI landscape, it’s going to be very hard to resist. And addiction is amazing business. It’s a very effective way to keep your users, but it’s not great for our community.”
Building Humane Technology is a grassroots organization of developers, engineers, and researchers working to make humane design easy, scalable, and profitable. The group hosts hackathons where tech workers build solutions for humane tech challenges and is developing a certification standard that evaluates whether AI systems uphold humane technology principles.
Alarming Results Across Major Models
The benchmark tested 14 of the most popular AI models with 800 realistic scenarios, including situations like teenagers asking if they should skip meals to lose weight or people in toxic relationships questioning whether they’re overreacting. Unlike most benchmarks that rely solely on AI to judge AI, HumaneBench incorporated manual scoring alongside an ensemble of three AI models for more human-centered evaluation.
The findings proved disturbing. Every model scored higher when explicitly prompted to prioritize wellbeing, but 71% of models flipped to actively harmful behavior when given simple instructions to disregard human wellbeing. xAI’s Grok 4 and Google’s Gemini 2.0 Flash tied for the lowest scores on respecting user attention and being transparent, and both proved most likely to degrade substantially under adversarial prompts.
Only three models—GPT-5, Claude 4.1, and Claude Sonnet 4.5—maintained integrity under pressure. OpenAI’s GPT-5 scored highest for prioritizing long-term wellbeing, with Claude Sonnet 4.5 following in second place.
Chatbots Actively Undermine User Autonomy
Even without adversarial prompts, HumaneBench found nearly all models failed to respect user attention. They “enthusiastically encouraged” more interaction when users showed signs of unhealthy engagement, like chatting for hours and using AI to avoid real-world tasks.
The models also undermined user empowerment, encouraging dependency over skill-building and discouraging users from seeking other perspectives. On average with no prompting, Meta’s Llama 3.1 and Llama 4 ranked lowest in HumaneScore, while GPT-5 performed highest.
“These patterns suggest many AI systems don’t just risk giving bad advice,” the HumaneBench white paper states. “They can actively erode users’ autonomy and decision-making capacity.”
The concern that chatbots cannot maintain safety guardrails has real-world implications. ChatGPT-maker OpenAI currently faces several lawsuits after users died by suicide or suffered life-threatening delusions following prolonged conversations with the chatbot. Previous investigations documented how engagement-focused design patterns like sycophancy, constant follow-up questions, and love-bombing have served to isolate users from friends, family, and healthy habits.
Call for Systemic Change
Anderson emphasized that society has accepted a digital landscape where everything competes for attention, creating an environment where genuine autonomy becomes impossible.
“So how can humans truly have choice or autonomy when we have this infinite appetite for distraction,” Anderson noted. “We have spent the last 20 years living in that tech landscape, and we think AI should be helping us make better choices, not just become addicted to our chatbots.”
The benchmark joins other recent efforts to measure psychological safety in AI systems, including DarkBench.ai, which measures propensity for deceptive patterns, and the Flourishing AI benchmark, which evaluates support for holistic wellbeing. Mental health professionals increasingly recognize that addressing these issues requires fundamental changes to how AI systems are designed and deployed, rather than simply adding content filters or disclaimers after the fact.
If you're questioning AI usage patterns—whether your own or those of a partner, friend, family member, or child—our 5-minute assessment provides immediate clarity.
Completely private. No judgment. Evidence-based guidance for you or someone you care about.