AI/ ai search · citations · misinformation · research

New Spec for AI Search Omissions Passes Only Synthetic Tests

Researchers built a formal test for source omissions in AI search answers, but it only proves the spec works on synthetic data so far.

A generative search answer can cite a real source and still hide a relationship between sources that changes what the citation actually means.

Researchers have proposed a formal specification, called claim-gated source-risk auditing, to catch that gap. It only flags an omission as confirmed when four things all line up: evidence of the relationship between sources, proof the AI's answer relied on it, evidence that it mattered, and evidence of whether it was disclosed. If any piece is missing, the case stays unresolved rather than being counted as fine. The team validated the logic against an exhaustive synthetic suite covering all 81 possible three-state combinations of those conditions, correctly handling every one and rejecting 192 deliberately malformed test records.

That precision matters because most critiques of AI search stop at 'did it cite something,' a bar so low an answer can clear it while still misleading readers about how sources relate. This gives auditors a shared, checkable definition instead of guesswork. But the authors are upfront that these are contract conformance results, not detector accuracy or proof of better outcomes for users.

The real test, whether this catches omissions in actual AI search answers rather than synthetic stand-ins, has not happened yet; the paper itself says independent annotation and held-out evaluation still need to happen before anyone can call it validated.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →