A new adversarial-attack technique borrows from AI training itself, but runs it backward.
A research team describes Inverse Knowledge Distillation (IKD), a plug-in for existing transfer-based adversarial attacks on image classifiers. Instead of pulling a prediction toward a target label, the way ordinary knowledge distillation does, IKD pushes a model's prediction on a manipulated image as far as possible from its own prediction on the clean version of that image. The loss function can be built from cross-entropy or KL divergence interchangeably - the paper shows the two produce identical gradients up to a constant, so the choice doesn't matter - but swapping in mean squared error instead noticeably hurts results, since it measures distance in output-probability space differently. Tested on ImageNet against convolutional networks, vision transformers, and models with built-in defenses, IKD improved how well attacks crafted on one model transferred to fool a different one.
Transfer-based attacks matter because they skip the need for direct access to a target system - an attacker trains against a stand-in model and hopes the trick generalizes, which is precisely the threat model for production image-recognition systems that don't expose their internals. The paper backs this up with an information-geometric argument tying IKD's gains to overlap between the stand-in and target model's sensitivity landscapes, which is a more rigorous explanation than the trial-and-error tuning that usually accompanies transfer-attack papers.
The code is public on GitHub, so anyone building or auditing an image classifier can go check whether their defenses hold up against it - academic adversarial-attack papers have a habit of aging well in a controlled benchmark and less well against a patched, determined, real-world target.