A world model trained on random physical exploration — no text, no labels — spontaneously organizes its internal representations around the geometry of space.
Researchers trained a VAE-based world model on embodied exploration and found its latent space developed spatial semantic structure that tracks physical geometry. Direction accuracy reached 0.677 versus 0.547 for a randomly initialized encoder. Position RSA hit 0.192 versus 0.029 for random encoders — a 6.6x improvement. Across 20 checkpoints, prediction performance and semantic alignment improved together (Spearman r=-0.61, p=0.004), suggesting a shared underlying driver rather than coincidence.
The finding matters because it shifts the grounding debate. For years, researchers argued about whether meaning requires language. This work suggests physical geometry is a sufficient organizing principle on its own — with direct implications for how we design embodied agents that need to reason about space without a text corpus propping them up.
There is a catch worth flagging: KL regularization strength controls whether any of this works. Set beta too high (0.1) and the encoder loses access to geometric structure; both prediction and semantic alignment collapse to near-chance by step 50,000. Drop beta to 0.001 and they recover together. That is a brittle knob — and one more reason to hold off on calling this a solved problem.