A research paper out this week proposes a way to catch hallucinations in retrieval-augmented generation at the sentence level, not just the answer level.
Retrieval-augmented generation — the approach of feeding a language model relevant documents at query time to ground its answers — reduces fabrication but does not stop it. Most existing detectors return a single score for an entire response, which tells you something went wrong but not where. The new method, called Grounding-Aware Sensitivity by Perturbation (GASP), fixes that by re-scoring each sentence in an answer under three conditions: with full context, with no context, and with each retrieved chunk removed one at a time. A sentence that collapses in likelihood when its source passage disappears is likely grounded; one that barely notices is likely hallucinated.
The practical upside is that GASP is training-free — it uses a threshold on the grounding features rather than a labeled dataset, which matters a lot for teams that cannot afford to annotate thousands of RAG outputs. Tested on three benchmarks with small instruction-tuned models (under two billion parameters each), it reached a response-level AUC of around 0.73 on RAGTruth and improved on perplexity and self-consistency baselines by clear margins. The one competitive alternative at the span level, a chunk-level entailment verifier, requires a separate model entirely.
The limits are worth noting: GASP transfers to TofuEval but stumbles on RAGBench's short-answer questions, where models can answer from memorized knowledge rather than retrieved text. That is an honest constraint — grounding sensitivity only works when the answer actually depends on the retrieved context. Most production RAG pipelines do fit that description, but knowledge-heavy Q&A does not.