A new study pitted AI models against humans in a race to click from one Wikipedia article to another, and nobody won decisively.
Researchers tested language models on the Wikipedia Game, where players must navigate from a starting article to a target using only the links on each page. They compared sentence embedding models and large language models across five Wikipedia languages and five academic literature citation datasets, built from Wikipedia and OpenAlex data, and released code so others can reproduce the graphs without relying on third-party APIs. The team benchmarked the models against human players and a simple TF-IDF baseline. No model significantly outperformed humans, though the best performers, GPT-o3 mini, GPT-4o, QwQ:32B, and DeepSeek-R1:70B, achieved comparable success, with performance and rankings varying by language and topic.
This is really a stress test for something that matters well beyond trivia: how well a model can retrieve information from a document collection too large to index outright, a problem that shows up anywhere an AI has to find a needle in a haystack of specialist linguistic or scientific text. The fact that performance varied by language and subject suggests these models do not have one uniform navigation skill. They have several, and some are weaker than others.
It is a useful reality check for anyone assuming frontier models have already solved open-ended information retrieval. They have not. They have just gotten good enough to keep pace with a person clicking blue links.