AI/ rag · ai-security · llama · research

Poisoned RAG Contexts Drop Llama 3.1 Accuracy From 77.9% to 43.5%

A new study finds a small quantized Llama 3.1 model's fact-checking accuracy plunges from 77.9% to 43.5% when all three retrieved passages are poisoned.

Feed a retrieval-augmented model bad documents, and it believes them almost half the time.

Researchers ran Llama 3.1 8B, a small quantized model, through a fact-checking task built from the FEVER dataset. They tested three ways to poison retrieved passages: swapping entities, swapping numbers, and inverting claims with negation. Across a sweep of 588 runs, they corrupted zero, one, two, or three of the three passages the model retrieved before answering. Accuracy dropped from 77.9% on clean context to 43.5% when every retrieved passage was tampered with.

The model didn't mostly invent new lies. Its main response to poisoned context was abstaining rather than fabricating, and a rough proxy for unsupported claims actually fell under attack rather than rose. That's a partial defense, but abstaining on a fact-check is still a wrong answer, and entity swaps flipped more correct answers than any other tactic tested.

The paper is small-scale and its automated grading is coarse, so treat the entity-versus-number contrast as a lead rather than a verdict - but the headline number stands on its own: corrupt every document a model retrieves, and it gets fact-checks right less than half the time.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →