AI agents that can click around a desktop still can't finish a CAD drawing.
Researchers built CADWorld, a benchmark of 200 tasks in FreeCAD, the free 3D-modeling software engineers use to design real parts. The tasks span 11 workflow categories, from basic sketching to assembly, manufacturing setup (CAM), structural simulation (FEM), and technical drawings. Agents work the way a person would: they look at screenshots and click, type, and drag through the FreeCAD interface. Success isn't judged on whether the screen looks right - it's checked against the actual saved file, including dimensions, constraints, and simulation results.
The results are rough. The best of seven agents tested finished just 17.5% of tasks successfully, compared with an 87.0% pass rate for a human expert given the same instructions. Weaker agents often crashed before saving anything usable; the stronger ones got further but still botched the structural and geometric details that make a CAD file actually useful downstream.
It's a reminder that clicking through software convincingly is not the same as producing an engineering file someone can trust with a real part.