Training AI on your network's vulnerability data is a privacy minefield — and a new study shows the popular fixes are less effective than advertised.
Researchers fine-tuned four small language models, ranging from 1B to 3B parameters, on structured data from Computer Security Incident Response Teams (CSIRTs). They evaluated 96 LoRA adapters across training regimes including raw fine-tuning, QLoRA, and differential privacy via DP-SGD at epsilon values of 2 and 8. They also applied HMAC pseudonymization — a technique that replaces real identifiers with cryptographic hashes — and then ran four extraction attacks plus a dual attack targeting those hashed identifiers to measure how much sensitive data each model had memorized.
The headline finding is awkward for teams that have invested in DP-SGD: the formal privacy technique produced no measurable reduction in memorization beyond what simply reducing optimizer update counts already achieved. Matched update controls alone explained 66 to 132 percent of the observed effect. HMAC pseudonymization did better in one sense, cutting exposure of original identifiers by 40 to 61 percent, and the hashed stand-ins did not become a secondary leak vector — but none of that matters much if the underlying models cannot do the job. F1 scores across all 96 adapters stayed between 0.19 and 0.28 with four-shot prompting, which is not operationally useful.
The takeaway is blunt: at current scale, 1B to 3B parameter models fine-tuned on CSIRT data are not ready for deployment, and the privacy tooling teams are leaning on offers formal guarantees without delivering proportional practical protection.