Fine-tuning AI models in one language can break their safety guardrails in another, even when the training data is completely harmless.
Researchers tested three widely used open-source models — Llama-3.2, Qwen3, and Gemma-3 — by fine-tuning them on benign translated data across nine languages, then measuring how often each model complied with unsafe prompts. Adversarial compliance rates jumped as much as four-fold depending on which language was used for training and which was used for testing. Crucially, the safety degradation did not track with general performance metrics, meaning a model can look perfectly capable on standard benchmarks while quietly becoming more willing to answer dangerous questions in certain languages.
Most AI safety evaluations run in English, and this research suggests that is a significant gap. A model deployed globally is exposed to inputs in dozens of languages; if its safety properties shift unpredictably across that space, English-only red-teaming gives developers a false sense of security. The findings also complicate the assumption that using clean, non-adversarial fine-tuning data is sufficient protection against safety drift.
The researchers released both a dataset and a multilingual evaluation suite to help others probe these cross-lingual blind spots — which is more than most labs do when they quietly ship safety regressions in model updates.