Researchers have built a lightweight probe that can flag whether a large language model is about to leak the system prompt or documents it was fed, before it even finishes generating a response.
The method, called LeakGauge, works by appending a suffix to a model's output that gauges its leakage behavior, then reading the token probabilities at that point to produce an attack-risk score. Rather than checking whether the model regurgitates the actual confidential text, the more reliable version of the technique asks the model to verbalize whether it's leaking anything - a content-agnostic signal that held up better across tests. The researchers ran LeakGauge against 11 different LLMs, including GLM-5.2 (753 billion parameters) and Kimi-K3 (2.8 trillion parameters), and reported AUROC scores between 0.944 and 0.996 on attacks the system hadn't seen during training. The signal stayed stable even when attackers switched languages or moved from verbatim copying to paraphrased disclosure.
That matters because prior leakage-detection work relied on probing a model's internal hidden states, which meant pulling apart the model's internals at inference time - not exactly practical for anyone running production infrastructure. LeakGauge instead works off something closer to the model's own output behavior, and the researchers say it can run as an input detector with fewer than 500 extra parameters and about 10 milliseconds of added latency. For companies stitching retrieved documents and system prompts into every LLM call, which is most of them now, that's cheap enough to actually deploy.
Worth remembering this is a single paper, not a shipped product, and AUROC numbers on known benchmarks tend to look rosier than performance against attackers actively trying to dodge the specific gauge being used.