Multimodal AI models can describe photos and answer questions about charts, but hand them a mechanical engineering blueprint and they fall apart.
Researchers introduced MechVQA, a dataset of 3,300 high-density mechanical drawings with 21,000 question-answer pairs, built to test and train AI models on real engineering documents. The benchmark spans 10 task types organized across three capability levels: Recognition, Reasoning, and Judging. Alongside the dataset, the team developed MechVL, a domain-specialized model trained in multiple stages on this material. MechVL outscored the best closed-source competitor by 7.57 percentage points on the benchmark's total score.
The gap matters because mechanical drawings are not decorative — they encode tolerances, projection rules, and geometric constraints that determine whether a manufactured part works or fails. General-purpose vision models miss these cues routinely, which limits AI's usefulness in actual design and inspection workflows where errors carry real cost.
A 7-point lead over closed-source rivals on a purpose-built benchmark is meaningful progress, though it also signals how far behind the starting line the whole field still is. When the benchmark itself has to be invented from scratch, that is less a breakthrough and more a baseline.