A new AI benchmark uses decades-old cartoons to expose how poorly today's models handle causal reasoning in video.
Researchers released CausalChaos!, a dataset built on clips from the Tom and Jerry cartoon series, designed to test causal question answering over video. The dataset presents models with "why" questions that require tracing multi-step cause-and-effect chains across character interactions and visual scenes. It includes multi-level answers — a short answer plus a detailed causal explanation — and adds a hard mode with deliberately confusing wrong answers designed to trip up models that rely on surface-level pattern matching. The work appears on arXiv and the dataset is available on GitHub.
Most video QA benchmarks test whether a model can identify what happened, not why. CausalChaos! forces models to connect a sequence of events into a coherent causal story, which turns out to be significantly harder. Cartoons are a smart choice for this: animators deliberately construct unambiguous cause-and-effect relationships to keep stories legible, which means the ground truth is unusually clean compared to real-world footage.
The researchers find current models perform reasonably on closed-ended questions but struggle with open-ended answers — a familiar pattern that suggests models are still better at picking from a list than reasoning from scratch. The gap between "can identify" and "can explain" remains the central unsolved problem in vision-language AI, and Tom and Jerry just made it measurable.