A new research benchmark argues that scoring AI creativity with a single number throws away the most useful information.
The Human Creativity Benchmark (HCB) gathers 15,000 judgments from domain professionals across five creative fields — covering ideation, mockup, and refinement phases. Instead of treating expert disagreement as a measurement problem to average away, HCB treats it as signal. Researchers collected pairwise preferences, scalar ratings on prompt adherence, usability, and visual appeal, and written rationale. They then split the results into two buckets: convergence, where professionals reliably agree, and divergence, where they legitimately do not.
The split matters because it maps onto two different expectations for AI tools. Technical correctness and visual hierarchy produce convergent judgments — things a model should simply get right. Aesthetic direction and conceptual risk produce divergent ones — things a model should stay out of the way on, deferring to the human using it. No model in the study performed uniformly well across all three workflow phases, which suggests that leaderboard rankings built on collapsed scores are hiding as much as they reveal.
Every major AI lab publishes benchmark numbers, and the temptation to roll everything into one tidy score is obvious — it makes press releases easier. HCB is a push in the other direction, toward evals that tell you what a model is actually good for rather than just how it ranks.