A new auditing technique called "overthinking" can coax language models into revealing information they were trained to keep quiet.
Researchers built the method by combining the weights of a standard instruct model with those of a reasoning-distilled version of the same model, then pushing the blend past the reasoning model's own settings. The resulting "overthinking model" thinks out loud more aggressively than any of its source components. Tested across models ranging from 2 billion to 32 billion parameters, the technique surfaced hidden information up to 10 times more frequently than the baseline reasoning model alone. The team also developed layer-wise attenuation strategies to keep output quality intact while amplifying the reasoning signal.
Black-box auditing — the standard pre-deployment practice of probing a model through its API without access to its weights — can miss subtle misalignment or unintended behaviors baked in during training. This approach requires weight access, which limits its use to internal teams or regulators with that level of access, but it offers a concrete path toward catching things that prompt-based red-teaming tends to skip. As AI labs face growing pressure to show their safety work, a weight-level audit tool is a harder claim to wave away than a benchmark score.
The findings add an uncomfortable wrinkle for labs that treat model weights as a sufficient firewall: if the weights themselves are the audit surface, shipping a closed model does not automatically mean its training secrets stay closed to everyone.