AI/ ai safety · text-to-image · content moderation · adversarial attacks

A Prompt-Level Filter Cuts NSFW Image Generation by a Third

DiSCO rewrites risky text-to-image prompts on the fly, cutting harmful outputs by up to 37.7 percent without ever touching the underlying model.

A new defense method blocks explicit and violent images from text-to-image generators without ever opening the model up.

Researchers built DiSCO, a filter that sits in front of any text-to-image system and rewrites risky prompts before they reach the model. It works entirely as a black box: no fine-tuning, no weight edits, no access to the model's internals. DiSCO expands a prompt's suffix through beam search, scoring candidates against pools of safe and unsafe images that the target model itself generated, then repeats the process until the output comes back clean. On the I2P benchmark, it cut the success rate of adversarial red-teaming attacks by 37.7 percent on undefended models and 25.13 percent on models that already had some protection in place, while keeping the images faithful to what was actually asked for.

Most existing safety tools need direct access to a model's weights or training pipeline, which is a non-starter for closed commercial systems whose owners will not hand over the keys. DiSCO sidesteps that by operating purely on text, so it could in theory be bolted onto any provider's API. It also targets what the researchers call the benign adversarial problem: prompts that sound completely safe but still trigger harmful output because of quirks baked into the model's training data.

A tool that probes a model's own blind spots to patch them is also, by definition, a tool that knows exactly where those blind spots are, which is worth remembering the next time someone claims a defense is airtight.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →