Researchers have proposed a way to stop AI-built knowledge graphs from quietly hallucinating their own facts.
The system combines a retrieval-augmented small language model with a branch of mathematics called formal concept analysis (FCA). FCA works as a symbolic verification loop: it proposes logical implications about concepts, and the language model either confirms them against retrieved sources or returns a counterexample. Every accepted implication, contradiction, and correction stays inspectable - the chain of evidence is explicit rather than buried inside model weights. The team tested the approach on rare ataxia data drawn from Orphadata, a structured disease database, seeding the system with as few as 10 starting attributes.
Ontology construction - the unglamorous work of deciding what counts as valid, structured knowledge - is exactly where language models tend to fail silently. An LLM asked to build a knowledge graph will produce something that looks coherent but may contain relations it invented. Tying each accepted claim to a retrieval source and running it through a formal verification step raises the bar for what gets committed as fact. For domains like rare disease research, where a wrong relation can cascade into bad clinical recommendations, that matters.
The numbers are modest: relation F1 scores of 0.29 to 0.52, and stricter implication F1 scores of 0.22 to 0.30. The authors are candid that identifying correct object-attribute pairs stays hard even with fixed candidate sets. This is early-stage research, not a deployed product - but it is a more honest approach to AI knowledge construction than most industry pipelines currently advertise.