AI explanation tools like SHAP and LIME turn into gibberish when they're pointed at Arabic, Hebrew, Urdu, or Persian text.
Researchers tested SHAP and LIME explanations on hate-speech and offensive-language classifiers built for four right-to-left languages. When they ran the rendered word labels back through OCR to check if they were even readable, character error rates ranged from 0.82 to 0.979, meaning the visible label often bore no resemblance to the actual token. A popular workaround that reshapes and reorders text made things worse for Urdu, pushing the error rate to 0.998. Upgrading to Matplotlib 3.11.0 flipped that workaround's effect entirely, breaking cases it used to fix. The team built SHAP-RTL, a rendering layer that corrects reading direction and letter shaping without touching the underlying attribution scores.
The math behind these explanations was never the problem; the bug lived entirely in how the results got drawn on screen. That distinction matters because anyone auditing an Arabic or Urdu hate-speech classifier for bias or accountability could have been staring at scrambled labels and drawing conclusions about the wrong words entirely.
It is a fair bet that explainable-AI tooling was built, tested, and screenshotted almost exclusively in English, and nobody checked whether the screenshots even worked in the other half of the world's writing systems.