Vision-language AI models are famously bad at picturing a room from the window, not the door - this paper tries to fix that.
Researchers built a framework called SpatialASM that trains large vision-language models to mentally simulate a scene from alternate viewpoints before answering spatial questions. It uses what the team calls an allocentric novel-view generator to produce plausible images of a scene from angles not in the original photo, grounded in the real visual evidence. That generator feeds two kinds of training data: step-by-step mentalization trajectories showing the model when to imagine a different viewpoint, and automatically built perspective-taking exercises that reinforce the skill. The team tested the approach on four different vision-language model backbones across seven spatial-reasoning benchmarks, including tasks that require comparing multiple images or reasoning about how a scene changes when the viewpoint shifts.
Spatial reasoning - knowing how objects relate to each other in 3D space, not just recognizing them - is one of the clearest gaps between these models and human perception, and it matters for anything from robotics to augmented-reality apps that need to track a scene from a moving camera. The more notable part isn't the accuracy bump itself; it's that the viewpoint-imagination training data is generated automatically rather than hand-labeled, which is normally the bottleneck for teaching models this kind of spatial chain-of-thought.
Benchmark gains in a research paper are not the same as a robot that stops bumping into furniture - the imagined viewpoints are still synthetic guesses, not real sensor data, and how well that holds up outside curated test sets remains an open question.