Researchers testing a standard noise technique as a privacy shield for AI hidden states found zero working mechanisms out of 1,536 tested.
The study examined whether neural network hidden states, the intermediate values computed between input and output, can be protected by adding Gaussian noise, a common statistical technique for obscuring data, without exposing model internals to a determined attacker. The answer is effectively no. A mathematical proof shows that any such release with acceptable utility leaves an exploitable direction, with the signal an attacker can follow growing linearly as the model gets wider. One mechanism, a diagonal inverse-Fisher release, does achieve strong privacy at every tested point across a 32-layer grid, but it sits at the usability edge rather than the safe middle that previous work assumed existed. A more sophisticated mechanism that appeared 13 times better under standard tests collapsed to 100% retrieval accuracy under adaptive attacks; separately, an attacker recovered 94% of GPT-2 prefixes from unprotected releases and 0% under the diagonal approach.
The findings undercut a common assumption in privacy-preserving AI: that you can bolt noise onto any existing model and get meaningful protection. The researchers propose a fix, but it requires building a new architecture from scratch. Their split-memory transformer maintains strong privacy-utility tradeoffs across model sizes from 30M to 1B parameters, while pretrained models adapted to use the best available mechanism top out at significantly weaker protection.
Most organizations deploying large models did not build those models. Telling them the solution requires starting over is technically correct and operationally useless.