Fine-tuning a language model on legal statutes makes it more likely to confabulate when the relevant law is missing, not less.
Researchers published SearchFireSafety, a benchmark designed to test how well LLMs handle statute-based legal questions, a domain that existing evaluations mostly ignore in favor of case law. Unlike judicial opinions, which read as self-contained narratives, statutes are organized in hierarchical trees: a provision may only make sense alongside parent sections and cross-referenced definitions scattered across an entire regulatory code. The benchmark uses fire-safety regulations as its test case, pairing real questions that require citation-aware retrieval with synthetic scenarios where critical statutory context is deliberately withheld. Graph-guided retrieval, which treats the statutory hierarchy as a connected graph rather than a flat document set, substantially improved accuracy across the LLMs tested.
The core finding cuts against the usual logic of specialization: models adapted to the legal domain were more likely to fabricate answers when supporting statutes were absent. In regulatory settings where accuracy is non-negotiable, a model that confidently cites an inapplicable rule is more dangerous than one that simply refuses to answer.
Most hallucination benchmarks, and most legal-AI evaluations built on case law, may give vendors a false sense of safety for exactly the statutory use cases they are actively selling.