A new benchmark just showed that AI models good at drawing bridges are bad at building ones that actually stand up.
Researchers introduced PolyBridgeBench, a benchmark that feeds multimodal large language models a visual scene and a set of engineering constraints, then asks them to generate a complete node-member-material bridge structure. The design first has to pass deterministic legality checks. Then it gets run through a physics simulator to see if it holds. If it fails, the model receives visual evidence from the failed rollout and a fixed, limited budget to attempt repairs. Across six MLLMs and 189 levels, the researchers separately measured structural validity on paper, actual success under simulated physics, and whether models could fix a design after watching it fail.
The gap between those first two numbers is the real finding: models regularly produced bridges that looked correct by the rulebook but collapsed once physics was applied. They were also highly sensitive to material budgets, and largely failed to repair their own designs after seeing them fail under the benchmark's strict-budget setting. That is a meaningful distinction for anyone pitching these models for actual engineering work rather than demos - passing a checklist is not the same as surviving load.
It is a useful, unglamorous check on a trend where multimodal models get credit for looking plausible. Understanding stress and failure is a different skill than generating a picture of a structure, and this benchmark is one of the first to actually separate the two.