Ask an AI model whether someone altered its internal wiring, and it draws a blank - even when the proof is sitting inside its own head.
A new framework called Open-Weight Masked Introspection (OWMI) tested eight open-weight models from seven different families. Researchers intervened on residual-stream sites, attention heads, and sparse-autoencoder features, then asked each model to report whether its computation had been changed. The answers were checked against sham runs, impact-matched random noise, and a text-only observer with no internal access. Across 78,000 measurements, no model's self-report beat chance - accuracy hovered around an AUROC of 0.5007, with the real effect bounded below 0.15 percentage points.
The strange part is that the information was there the whole time. A version of one model fine-tuned specifically to spot these interventions recovered them almost perfectly, and a simple linear probe reading the same activations hit up to 95.8% accuracy, with zero errors right before the model opened its mouth. That gap between what sits in a model's internal state and what it says out loud is a problem for anyone hoping to use a model's own testimony as an oversight tool.
One wrinkle: in a single model, the yes-or-no answer never changed, but the confidence attached to it did - separating real interventions from shams at an AUROC of 0.647. So the signal can leak out sideways even when the stated answer stays flat. The researchers note this is a snapshot of today's open-weight models, not a verdict on what future, more capable systems might manage - but for now, trusting a model's word is not a safety strategy.