A new benchmark says AI still can't reliably tell you if your squat form is bad.
Researchers built FitAQA, a benchmark for testing multimodal AI models on fitness form-checking. It includes 2,219 workout videos and 5,512 question-and-answer pairs spanning 30 bodyweight exercises. Working with sports-science experts, the team built a taxonomy of 38 recurring form errors sorted into six categories: alignment, symmetry, stability, coordination, tempo, and completeness. The benchmark then tests models on three separate skills: spotting the relevant visual evidence, judging whether a movement was correct, and pinpointing when in the video a form error happened.
The results aren't flattering. Current multimodal models struggle to assess exercise quality comprehensively and are especially weak at localizing exactly when a form error occurs. The more telling finding: when researchers fed models the correct visual evidence directly, judgment accuracy improved substantially, meaning the models' weakness isn't reasoning about fitness, it's actually seeing what's happening in the video.
That's a useful, unglamorous distinction for anyone building an AI personal trainer: the bottleneck is computer vision, not domain knowledge, and it's not solved yet.