AI image generators are confident fabricators, and researchers now have 20,000 prompts proving it.
A team published SearchGen-20K and SearchGen-Bench, a dataset of 20,839 prompts across 22 domains designed to expose what image generators get wrong when asked about new characters, trending entities, or anything that postdates their training cutoff. Frontier open-source generators scored between 21 and 28 out of 100 on the benchmark — a failure rate existing evaluations never surface. The researchers call this the "knowledge boundary": the structural divide between what a model absorbed during training and what it must pull from somewhere else. Bolting on a search tool naively made things worse, injecting irrelevant retrieved content into prompts the generator already handled fine.
The fix is a co-training approach they call teach-then-search, which teaches the model to recognize when it does not know something before deciding whether to retrieve. That self-awareness matters because the knowledge boundary is not fixed — it shifts as the world changes, which is precisely the problem search alone cannot solve. Even a minimal version of the method produced steady improvement, and the team frames it as a foundation for recursive self-improvement.
The benchmark and corpus are released as a replayable offline harness, which is the right call — reproducible evaluation is the piece most "agentic AI" research skips entirely.