An arXiv paper describes a two-part system designed to make AI mental health responses safer and more clinically sound.
The research team built TheraJudge, an open-source model trained on human-annotated data to rate therapeutic responses across seven psychological dimensions — including Safety, Relevance, and Empathy. It achieved intraclass correlation coefficients of 0.87 to 0.95 against clinician ratings, beating both supervised baselines and proprietary judge models. They then built TheraAgent, a multi-agent system with Critic, Coach, and Therapist roles that uses TheraJudge's scores as a feedback signal to revise responses. Under blind evaluation, TheraAgent lifted human-rated therapeutic quality by 0.43 points on a 5-point scale, with 96% clinician inter-rater reliability.
The headline result is what happened to the worst outputs: responses scoring 3 or below improved by 2.45 points, with a 94% recovery rate. That matters because low-quality AI mental health responses aren't just unhelpful — they can be actively harmful, and most evaluation frameworks treat quality as a reporting metric rather than something to act on.
The code is open-source on GitHub, which is worth noting given how much AI mental health work happens inside closed commercial products with no external audit trail.