AI/ ai · video-generation · reward-models · reinforcement-learning

A Video AI Reward Model That Grades With Rubrics, Not Vibes

RewardVerse scores AI video against explicit criteria instead of one opaque number, aiming to fix the unstable reward signals that plague video-generation RL.

A new paper proposes grading AI-generated video with written rubrics instead of a single gut-check score.

Researchers behind RewardVerse built a reward model for video generation that first drafts explicit evaluation criteria for a given prompt, then scores the video against those criteria rather than guessing a number outright. The training process, called Rubric-Guided Policy Optimization, runs in two stages: it warms up the scorer using self-generated seed rubrics, then jointly trains a rubric generator and scorer so the criteria adapt to each query while staying aligned with human ratings. The team tested the system on EvalVerse, a benchmark spanning 16 quality dimensions, plus outside datasets. They report the rubric-based approach curbs what they call scalar drift - the tendency for a reward model's scoring scale to shift or collapse across different prompts - and claim state-of-the-art results on both single-video and head-to-head video comparisons.

Reward models are the referee for reinforcement learning in video generation: get the scoring wrong and training drifts toward whatever the model can game, not what actually looks good. A rubric that adapts per query is a sensible answer to a problem RLHF researchers have wrestled with for years in text models, where unstable or exploitable reward signals produce over-optimized, hollow outputs.

The paper does not name the specific prior reward models it claims to beat or publish the score margins on EvalVerse, so "state-of-the-art" is a claim to take on faith until the numbers or code show up.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →