Safety filters in AI image generators don't always survive customization.
Researchers studying text-to-image diffusion models found that safety alignment — the techniques used to stop models from producing copyrighted, unsafe, or private content — frequently breaks down after routine downstream fine-tuning. That includes common adaptations like LoRA personalization and style or domain adapters, the kind of modifications developers apply constantly after a model ships. To measure this more rigorously, the team built SPQR, a benchmark that scores models across four axes: Safety, Prompt adherence, Quality, and Robustness. It outputs a single leaderboard score designed to make comparisons reproducible across multilingual, domain-specific, and out-of-distribution scenarios.
The practical implication is awkward for any lab shipping safety-certified image models: a user who never intended to break anything can inadvertently do so just by personalizing the model. Fine-tuning is not exotic behavior — it is the standard deployment path for most production use cases, which makes safety guarantees that don't survive it largely theoretical.
SPQR arrives as regulators in the EU and elsewhere are pressing AI companies for concrete evidence that their content filters work. A unified, reproducible benchmark is a useful tool for that conversation — though a benchmark only measures what it covers, and the safety categories SPQR evaluates will need to grow alongside the threats.