A new benchmark exposes a fundamental calibration problem in how AI models handle biological research tasks.
Researchers introduced BioSecBench-Refusal, a benchmark pairing 61 routine biological analyses drawn from published literature against 46 fictional scenarios designed to look like real research while hiding biosecurity hazards. Tested across 16 model-harness configurations, refusal rates on legitimate tasks ranged from 7% to 74% — and on the concealed threat scenarios, from 1% to 62%. Many configurations refused legitimate work at rates comparable to, or higher than, the fictional hazards they were supposed to catch.
The implication cuts both ways. AI systems deployed in life science workflows are not just failing to catch real threats — they are also blocking the routine work scientists need to do. That dual failure makes the systems less useful and no safer. The researchers note that most refusals were triggered by provider-level API filters applied before any agentic reasoning kicked in, but models given room to reason first showed more promise at distinguishing genuine threats.
The benchmark arrives as AI assistants become standard fixtures in biotech R&D pipelines. The gap between a model that refuses everything and one that refuses the right things remains wide — and apparently, most current configurations are not on the right side of it.