A new paper proposes catching unsafe chatbot replies not by reading them, but by watching the math underneath them move.
Researchers extended a dynamical-systems framework originally built for detecting AI hallucinations and repurposed it for safety classification. The method projects both the prompt and the model's response into high-dimensional embedding spaces, then fits separate Koopman-based predictive models - one trained on "safe" interaction patterns, one on "unsafe" ones. A new response is scored with a differential residual: how much worse the safe model predicts its trajectory compared to the unsafe model. Tested as a black-box classifier across three safety benchmarks and three embedding models, adding the prompt's embedding dynamics - not just the response's - improved detection of violations that depend on the back-and-forth between prompt and reply, especially with causal decoder models like Llama-3, while purely response-based violations did better with dense semantic embeddings.
Most content moderation classifiers just read text and guess. This one tracks how a model's internal representation evolves while generating an answer, treating a jailbreak as a shift in trajectory rather than a bad word choice. That framing could resist simple phrasing tricks, since it watches the shape of the response rather than its vocabulary - though the paper only reports offline benchmark results, not adversarial red-teaming against motivated attackers.
Every new detector claims strong recall on today's jailbreaks; the real test is whether attackers can adapt their embeddings as fast as they adapt their prompts.