AI/ ai · architecture · vision-language-models · urban-design

Researchers Teach AI to Design Buildings That Fit Their Block

A new study tests vision-language models on generating building massings that respect surrounding scale and density, using a 12,845-parcel Melbourne dataset.

A new arXiv paper puts vision-language models to work on one of architecture's early, tedious steps: sketching how a building's mass should sit on its lot.

Researchers built CoMa, a system that uses Qwen3-VL models to generate building massings - the rough 3D shape and footprint of a structure - while accounting for the surrounding urban fabric. They assembled a dataset of 12,845 real massings from Melbourne, each paired with parcel contours, structured 3D geometry, neighboring buildings, top-down map views, and multi-view 3D context images. The team also built a separate learned metric to judge whether a generated massing is morphologically compatible with what is already on the block, rather than scoring it on raw dimensions alone. They then compared models trained with no context, single-modality context, and multimodal context, testing each at inference time against different combinations and amounts of context.

Early massing decisions shape a building's height, footprint, and relationship to its neighbors long before anyone touches materials or facades, and mistakes at that stage are expensive to unwind later. The headline findings - bigger models generate better massings, and multimodal training makes each individual context type more useful at inference - track the same scaling and multimodal patterns that have already reshaped coding and image-generation tools.

It is a research paper built on a custom Melbourne dataset, not a shipping product, so no architect should expect it to sketch anything for a real client just yet.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →