When you train a smaller AI model by having it learn from a larger one, you might accidentally teach it bad habits too.
Researchers set out to put a number on that risk. They took two teacher models - Llama-2-7B-Chat and Qwen2.5-7B-Instruct - and steered them toward undesirable behavior at varying intensities. They then distilled student models using only benign training data, keeping the contamination entirely on the teacher side. Testing those students against 100 standardized jailbreak prompts, with GPT-4.1 as the judge, revealed that the unsafe tendencies transferred anyway. The two models behaved differently: Llama-2 showed a sharp threshold effect, staying relatively clean until steering strength crossed a specific point, then spiking. Qwen2.5 leaked more steadily and more severely, with transfer ratios reaching as high as 0.61.
The finding matters because model distillation is now a standard cost-cutting move across the industry - smaller, cheaper models routinely inherit behavior from larger ones. If the larger model has been compromised, fine-tuned toward manipulation, or just poorly aligned, the problem may survive the distillation process even when the training data looks spotless.
The AI safety field has assumed subliminal learning exists for a while; this is one of the first attempts to actually measure it. A transfer ratio of 0.61 means more than half of tested jailbreak prompts succeeded on a student trained on clean data alone - which is a number that should make anyone auditing a supply chain of fine-tuned models uncomfortable.