Security/ ai-safety · jailbreak · mixture-of-experts · llm-security

Researchers Strip Safety Refusals From a 320B AI Model

A jailbreak once proven only on small models now strips refusals from a 320B mixture-of-experts AI, but only when three weight types are edited together.

A single direction buried in a 320 billion parameter AI model's weights is enough to erase its safety refusals, and researchers did not need to train anything to find it.

A team applied a known jailbreak method called directional ablation to GLM-5.3-Flash, a 320B parameter mixture-of-experts model with 288 routed experts and FP8 quantized weights. The technique, previously proven only on dense models up to about 70B parameters, isolates a refusal direction from a few hundred contrastive prompts and subtracts it from the weights, no gradient training required. Editing the attention, dense, or routed-expert weights on their own barely dented refusal behavior, cutting it by at most 15 percent. Editing all three together removed 78 percent of refusal, and 74 percent of that effect only appeared once the edits were combined.

That gap matters because it shows defenses built for smaller, simpler architectures do not automatically carry over to today's mixture-of-experts models. The standard recipe, which edits modules by name, reached only 8.5 percent of the total effect on this model, so a defender following the old playbook would conclude the model was still safe when it was not. Across seven harmful-content benchmarks, the attack cut refusal rates by 41 to 89 percentage points with no measured drop in other capabilities.

Not every category folded. Subspaces tied to violence, sexual content and hate held onto some refusal at every rank the researchers tested, a reminder that erasing safety training is not a single on-off switch even when the attack mostly works.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →