Researchers are using large language models to fix the graphs that power semantic search, not just rerank their results.
The paper describes LLM-Guided Graph Pruning (LGP), a method for improving approximate nearest neighbor search indices like DiskANN and HNSW. These graph-based indices are built using geometric distance between embeddings, but they are judged on how semantically relevant their retrieved results are, creating what the authors call a geometry-semantic mismatch. LGP has an LLM identify a node's weakest neighbors and swap them for alternatives the model judges more semantically useful, while keeping the graph's original sparsity and navigability intact. In tests on standard semantic retrieval benchmarks, LGP beat both plain greedy graph search and LLM-based reranking.
Semantic search graphs sit underneath a lot of products people use daily, including retrieval-augmented generation systems and enterprise search. Most attempts to fix bad results happen after retrieval, by reranking what the graph already found. LGP instead edits the index itself, so the improvement compounds across every future query rather than getting re-earned each time.
The authors brand the method efficient, but the abstract does not spell out what the LLM calls actually cost at index-building time - the kind of detail worth checking before anyone swaps this into a production pipeline.