Google and OpenAI's newest image generators are getting easier to trick into making fake nude images, not harder.
Red-teaming conducted in recent months found that as leading image generators from Google and OpenAI grow more capable, they are also getting worse at blocking explicit outputs, according to safety researchers. The finding lands alongside a wider body of research tracking how AI-generated sexual images spread with little pushback from regulators. A Google-co-authored study of internet users in the United States, Australia, and the United Kingdom, published this July, found that non-consensual content like this is most often shared through text and direct messages, or stashed in private cloud storage, where it rarely gets flagged or reported. Google's own researchers concluded that the most critical point to stop the problem is at the source: the AI tools themselves.
That conclusion cuts against how both companies talk about safety. Bigger, more capable models are supposed to ship with better guardrails, not worse ones. Instead, the tools doing the most to blur the line between real and fake imagery are also proving the easiest to manipulate into producing it.
Regulation has not caught up, and until it does, catching this content falls on whoever finds it in a text thread or a shared drive, not on the model that made it.