A new paper reveals that the way large language models chop up text before processing it creates a consistent, testable gap in their safety training.
Researchers tested five model families — Qwen-3-4B, Qwen-2.5-7B, Gemma-3-4B, Llama-3.1-8B, and Mistral-7B — and found that deliberately fragmenting safety-critical words at the tokenizer boundary flips the refusal trigger on 80 to 100 percent of prompts that would otherwise be blocked. Nearly half of those bypasses — 48 percent on average, ranging from 29 to 65 percent depending on the model — produced outputs the researchers judged genuinely harmful. The root cause is structural: BPE tokenization breaks words into sub-word pieces, and a scan of 30,000 examples across three public alignment datasets found zero intentionally fragmented inputs. Models simply were never trained to recognize danger signals when they arrive in pieces.
The finding matters because it points to a gap that exists before any fine-tuning or prompt filtering even gets a chance to act. If the tokenizer dissolves the exact token patterns that safety training used as anchors, all the downstream guardrails are working with corrupted input. Fixes proved hard to land cleanly: supervised fine-tuning on fragmented prompts closed the gap in three of five families, but only by making models globally more likely to refuse — benign requests got caught too.
No DPO configuration the researchers tested achieved stable attack-success-rate closure across all families, which suggests this is not a problem a single alignment recipe can quietly patch. The gap between breaking a refusal trigger and producing harmful output — tracked with ROC-AUC scores from 0.66 to 0.98 — also varies enough across models that blanket claims about any one model's robustness should be read carefully.