AI/ ai · open-source · security · model-safety

A New Test Flags Stripped Safety Filters in AI Checkpoints

Researchers built a two-signal audit that identifies abliterated open-weight models before deployment, catching 53 of 57 known cases in a 273-checkpoint test.

A new pre-deployment audit method can flag open-weight AI models that have had their refusal mechanisms removed - and it does not need to run the model to do it.

Researchers tested a combination of two internal signals on 273 checkpoints spanning Qwen, DeepSeek-distilled Qwen, Llama, and Gemma families. The first signal measures an activation gap between a candidate checkpoint and a reference model; the second measures how much the weights have shifted from the base. Neither signal alone is reliable, but together their combined z-score reached an AUROC of 0.95 - separating 57 known abliterations from 37 benign fine-tunes and merges. A threshold derived from that dataset transferred to held-out model families at 89 percent balanced accuracy, missing only four cases.

The gap matters because runtime content filters score outputs, not the model artifact itself. A platform can deploy a stripped checkpoint, run it through a standard safety evaluation suite, and get passing marks if the model is clever enough about its generations. An artifact-level audit closes that window - at least partially.

The paper is careful to call this triage, not a lock. The researchers document two failure modes in order of severity: a spoofed reference model can defeat both signals without any training at all, and a determined actor who controls the checkpoint can train it past the detection threshold while keeping it guard-unsafe and coherent. The first failure is the more troubling one - it requires no compute, only a crafted reference. As open-weight model hosting scales up, an audit that depends on an attested reference is only as strong as whoever controls that attestation.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →