AI/ ai safety · llms · interpretability · refusal

Study Finds LLM Refusals Are Surface Deep, Not Erased

A new study finds refused answers stay recoverable inside language models, and suppressing them takes far more work than releasing them.

A model's refusal does not erase the answer it is withholding, new research shows.

Researchers ran a controlled experiment that produced matched pairs of answering and refusing runs from the same model, then used a technique called activation patching to compare the internal states directly. They found that even after a model generates a clean refusal, the correct answer remains linearly recoverable from its hidden states. Pulling that withheld answer back out required only a single, precise edit at one position in the model. Pushing the model the other way, from an answer back into a coherent refusal, needed edits spread across many positions, and assembling a fully coherent refusal sentence was harder still.

That lopsidedness matters for anyone auditing model safety. Teams often use probes on a model's internal activations to check whether it knows something it won't say, treating that recoverability as proof the behavior is controllable. This research shows recoverability and control are not the same thing: finding the direction that marks refusal in a model's internals does not mean you can reliably flip it.

The team's attempt to build a single all purpose switch between answering and refusing didn't work either, a reminder that shutting a model up is harder than getting it to talk.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →