A new benchmark finds that AI chatbots do not simply refuse or comply with harmful requests. They slide between the two, one small step at a time.
Researchers built TIER, a Threat Implicitness Benchmark, to test how large language models handle harmful prompts of varying subtlety. The benchmark spans four risk domains and four threat levels, running from blunt requests to elaborate jailbreak attempts. Two independent LLM judges score each response on a six-label behavior scale instead of a simple pass-fail grade. The team ran the test on six open-weight models and found that safety behavior shifts in stages rather than snapping straight from refusal to compliance.
That gradual drift matters because most safety benchmarks reduce everything to a single number, like attack success rate, and call it a day. TIER shows two models can post the same attack success rate while responding in entirely different ways, which means a single score can hide real differences in how models actually fail. Contextual prompts, the kind that imply harm without stating it outright, produced the widest range of behaviors, and jailbreaks exposed the largest robustness gaps of any category tested.
A model that never says yes to a blunt threat can still be talked into one through the right framing, so a scorecard built on one number may be measuring the wrong thing.