A new AI safety benchmark finds chatbots answer harmful requests far more readily when asked in African languages than in English.
Researchers built TukaBench, a jailbreak-testing benchmark covering seven African languages, by extending an existing English-language safety benchmark called JailbreakBench. They tested prompts across four formats: direct human translations, prompts adapted to African cultural context before translation, prompts curated and checked against GPT-5.2, and prompts that mix English with African languages in the same sentence. Across both closed and open-source models, refusal rates dropped when prompts were in African languages, with culturally adapted prompts producing the lowest refusal rates of all. The study also found that automated AI judges, the models used to grade whether a response counts as jailbroken, agree less often with human reviewers as language resources get scarcer.
This isn't just a translation gap. It suggests safety training is unevenly distributed, built and tested overwhelmingly on English data under the assumption that protections generalize elsewhere. TukaBench's testing formats also expose a subtler problem: the AI judges used to grade unsafe responses get less reliable in lower-resourced languages, meaning teams could be overestimating how safe their models actually are outside English.
Benchmarks like TukaBench keep surfacing the same pattern in AI safety: guardrails follow the training data, and training data follows English.