A new benchmark says AI models can describe a map well but can't reliably redraw it.
Researchers built GeoSpatialBench, a benchmark that tests multimodal AI models on spatial reasoning using real urban data from six geographically diverse cities. It covers 28 topological, directional, metric, and ordering relations, presented both as structured GeoJSON data and as map images, with every ground-truth answer checked through GIS operations. Unlike prior spatial benchmarks, it doesn't stop at question answering - it also asks models to edit a layout while staying inside stated geometric constraints. Running frontier MLLMs through both tasks exposed a split between recognizing spatial facts and acting on them.
The gap is more specific than a vague "AI struggles with space" story. Models handle structured GeoJSON noticeably better than the same information rendered as a map image, and combining both formats offers no real advantage over GeoJSON alone - multimodal input isn't a free upgrade here. The sharper problem shows up in editing: performance on ordering relations drops compared to plain question answering, and most failed edits break at least one of the constraints the model was explicitly given.
That distinction matters for anything billed as an "embodied" or "world model" AI system - reading a map and correctly changing it turn out to be very different skills, and right now most models only have the first one.