Video AI models have a cheating problem: they guess the action from the object, not the movement.
Researchers studying zero-shot compositional action recognition found that models trained to identify verb-object pairs — think "open drawer" or "cut onion" — routinely skip the verb entirely. Instead of reading motion over time, they identify the object and guess the most statistically common action associated with it. The paper traces this to two causes: sparse supervision for rare verb-object combinations, and an imbalance in how the model weights verb versus object signals. The result is a model that looks confident until you hand it an action it has never seen paired with a familiar object.
This matters because real-world deployment requires exactly the generalization these models fail at. A household robot or a video moderation system cannot be pre-trained on every possible verb-object combination; it has to reason from parts. If the model is just matching objects to their most frequent training-set verb, it is not reasoning at all — it is memorizing.
The proposed fix, called RCORE, has two components. Co-occurrence Prior Regularization deliberately surfaces rare verb-object combos and treats the frequent co-occurrence patterns as negatives to push against. Temporal Order Regularization for Composition forces the model to be sensitive to the sequence of frames, not just which object appears. Both interventions are tested on established benchmarks, where they reduce the shortcut behavior and improve generalization. Whether the gains hold outside controlled benchmarks — and against the full chaos of real video — is the question that always follows a paper like this.