A research team has built a cross-domain framework to measure whether large language models are actually creative — or just good at sounding like it.
CreativityPrism consolidates eight tasks across divergent thinking, creative writing, and logical reasoning into a single benchmark, scoring outputs on quality, novelty, and diversity. The researchers tested 17 state-of-the-art models and found that frontier-scale LLMs outperform smaller, locally-deployable open models by about 0.10 points — roughly 15% — on creative writing and logical reasoning. But in divergent thinking, the gap disappears entirely. Automated judges, validated against human annotations, handle scoring so the framework can scale without bottlenecking on human reviewers.
The divergent thinking result is the one worth sitting with. That domain — generating a wide range of original ideas from a prompt — is exactly where human creativity is hardest to fake, and it is also the area least targeted by current post-training pipelines. The finding suggests that labs optimizing for benchmark-friendly outputs may be leaving a meaningful dimension of creativity on the table.
The study also found that strong performance in one creative dimension rarely transfers to others, with novelty scores showing weak or even negative correlations with other metrics — a result that should give pause to anyone treating a single leaderboard score as proof that a model is genuinely creative.