A research team says it can filter bad LLM prompts faster than existing tools — and without touching the model weights.
kNNGuard works by tapping the hidden activation layers of an existing large language model rather than training a separate classifier on top of it. Given a labeled bank of just 50 safe and unsafe example prompts, the system runs a multi-layer k-nearest-neighbor search that fuses activation-space and embedding-space scores to decide whether a new prompt is safe. The researchers tested it across six domains covering both topical filtering and adversarial security prompts, and reported F1 scores competitive with or better than fine-tuned guardrails. It runs 2.7 times faster than the best comparable guardrail and 10 times faster than a fine-tuned safety classifier.
The speed gap matters because latency is the silent killer of production guardrail adoption — teams often skip or thin out safety layers when they slow responses noticeably. Swapping in a labeled bank of 50 prompts in under 10 seconds also means domain adaptation is cheap enough to do on the fly, which is a real operational advantage over methods that require retraining every time scope changes.
Most guardrail research races to build bigger fine-tuned classifiers; kNNGuard is a bet that the safety signal is already sitting in the activations of models teams are running anyway — and that you just need to know where to look.