Researchers have built a new scoring system that measures whether an AI's creative output is actually original, not just plausible.
The method, described in a paper on arXiv, tackles a familiar problem: large language models writing poetry, prose, or math solutions tend to converge on safe, predictable answers. The usual fix is cranking up the "temperature" setting to introduce more randomness, but that often produces sloppier, less coherent results. The researchers instead built a context-based score, grounded in information theory, that rewards outputs for staying accurate to the prompt while diverging from what the model would typically produce. They then used that score as a reward signal to fine-tune models with reinforcement learning, testing the approach on tasks like poetry generation and math problem solving.
The trade-off between originality and quality has dogged generative AI since the first chatbots learned to write cover letters that all sound the same. Most attempts to fix it tinker with sampling settings at inference time, a blunt instrument that treats every prompt the same way. Training a model to directly optimize for a measurable originality score, rather than hoping randomness accidentally produces something good, is a more targeted approach, and one that could matter for any product selling itself on AI "creativity."
Whether this scales beyond controlled benchmarks like poetry and math problems, into the messier world of marketing copy or brainstorming, is the open question, and the one that will determine if this is a genuine fix or another metric that looks good on a paper and does little in production.