AI/ ai · knowledge-graphs · nlp · research

LLMs Can Draft Domain Ontologies, but Experts Still Fix Them

A new study tested GPT-3.5 and GPT-4 on building knowledge structures for a niche maritime domain — the results were coherent but not production-ready.

Researchers found that large language models can rough out domain ontologies, but human experts still need to clean up the work.

A paper out of Brazil tested GPT-3.5 and GPT-4 on a specific task: automatically constructing ontologies for the country's maritime territory, a domain researchers call the Blue Amazon. The team generated twenty ontologies and then had human domain experts evaluate them. Both models produced conceptually coherent hierarchies, but not a single output was deemed satisfactory on its own — every one required further refinement before it could serve as a real reference.

Ontologies are formal knowledge structures that let both humans and software systems share a common understanding of a domain. Building them by hand is slow, specialist work, which is why so many niche fields simply go without them. If LLMs can accelerate that process — even as a first draft — the practical upside for data interoperability and knowledge management is real.

The catch is familiar: models trained on general text tend to produce plausible-sounding structures that still miss domain-specific nuance that only a practitioner would catch. That pattern has shown up repeatedly in LLM-assisted knowledge engineering, and this study adds a data point from a domain narrow enough that training coverage is likely thin. The more honest framing may be that LLMs are good at starting conversations with subject-matter experts, not replacing them.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →