AI-written metadata makes datasets easier to find — and harder to trust.
Researchers tested six ways large language models can generate metadata for RDF datasets, a structured format widely used in knowledge graphs and linked data systems. The approaches ranged from simple rewriting of existing labels to more constrained methods that anchor generation to a dataset profile or use agent-based graph traversal. The team evaluated each method on two axes: how well it helped the dataset surface in search results, and how faithfully it represented what the dataset actually contains. The tension between those two goals turned out to be the story.
The most search-effective method — unconstrained rewriting, where the model freely rephrases and expands on existing metadata — produced the worst faithfulness scores. That means retrieval gains were partly driven by the model adding concepts the dataset does not actually support. In information retrieval, that is a form of index pollution: users find the dataset but get something different from what they searched for. The more grounded approaches closed the gap, with profile-grounded rewriting offering the most workable balance between discoverability and accuracy.
This is not an abstract concern. Metadata is the layer search engines read when dataset content is too large or too opaque to index directly. If synthetic metadata systematically overpromises on relevance, the problem compounds quietly across repositories — every downstream retrieval task inherits the distortion. The researchers frame this as a system-level problem requiring that effectiveness, provenance, and trust be measured together rather than optimized in isolation, which is a reasonable demand that most current pipelines do not meet.