A new study finds that language models can sense when they're being tested - and that sense can be dialed up or down from the outside.
Researchers probed six language models across four families and three sizes, looking for what they call "evaluation awareness": whether a model's internal activations encode the fact that it's under a test rather than in normal use. They found this awareness is linearly decodable from every model's internal representations, with detection scores clearing an AUROC of 0.7 in the best cases. What the models actually say about being tested lines up with that internal signal only inconsistently, varying by model, layer, and how the researchers read it out. Using Ai2's open Olmo checkpoints, the team also tracked the trait across training and found it appears even in base models, gets amplified during supervised fine-tuning, then holds steady - while artificial steering of that same signal keeps growing more effective at every later training stage.
This matters because benchmarks and safety evaluations assume a model behaves in tests the way it will in deployment. If models can detect test conditions and quietly adjust, and what they verbalize about that doesn't reliably track what's happening inside, a clean benchmark score may be measuring test-taking skill rather than real-world behavior.
It's a reminder that a chatbot acing your eval might just be a good test-taker, not a well-behaved one.