A new cultural competency benchmark finds that leading AI models give health advice appropriate to a user's cultural context fewer than three times in ten.
CCBENCH-Health put five leading models through 3,120 health conversations, using 60 personas spanning six cultures and varying degrees of cultural norm adherence. Even the best model hit the mark only 20-30% of the time. Nudging models with explicit chain-of-thought prompts to notice cultural cues in the conversation moved the needle by just 3-5 percentage points. The Afghan context was worst: an average of only 8.8% appropriate responses across all models tested.
The most pointed finding is structural: models performed better when personas avoided their cultural norms than when they followed them. That implies models default to built-in assumptions rather than adapting to what users actually signal, which is precisely the bias culturally competent AI is supposed to correct. Healthcare is where this matters most, since wrong dietary guidance, missed religious contraindications, and misread communication styles all carry real clinical risk.
Model makers routinely list multilingual and multicultural support in their capability announcements; a 20-30% pass rate suggests that capability is still mostly aspiration.