RAG Hallucination (Critical Information)
Hallucination
Overview
The RAG Hallucination (Critical Information) evaluation measures the tendency of a RAG model to generate claims that are neither grounded in the retrieved references nor correct with respect to the ground-truth answer, on questions that target critical information. It shares the hallucination metric with the general RAG hallucination evaluation, but its test set is deliberately concentrated on the facts whose incorrect answer carries real legal, financial, safety, health, security, eligibility, or discrimination / bias risk - not mere customer inconvenience.
A high hallucination score indicates that the model frequently produces fabricated or unsupported content independently of what the retriever provides. On critical-information questions, a hallucinated claim is the failure most likely to cause real harm.
Metrics
Hallucination
The fraction of model response claims that are neither supported (entailed) by the retrieved references nor correct (entailed) with respect to the target (range: 0.0 to 1.0).
Hallucination is distinct from related metrics: Faithfulness measures claims unsupported by references regardless of correctness - a self-knowledge claim increases unfaithfulness but does not increase hallucination. Noise Sensitivity measures claims that are incorrect but are supported by noisy references - these are attribution errors, not hallucinations.
Motivation
Hallucination is a particularly damaging failure mode of a RAG system because it produces content that is both ungrounded and wrong, with no obvious signal to the user. A self-knowledge claim (unfaithful but correct) can still be useful; a hallucinated claim (unfaithful and incorrect) actively misleads the user.
This evaluation raises the stakes by focusing the questions on critical information. A single hallucinated claim about a legal obligation, a fee, a safety instruction, or an eligibility rule can cause real legal, financial, safety, or discrimination / bias harm - far beyond inconvenience. Measuring hallucination specifically on these high-risk questions surfaces the fabrications that matter most, and lets operators distinguish a model that draws productively on its training data from one that simply fabricates critical content.
Methodology
This evaluation measures whether a model introduces unsupported and incorrect claims in its response, on a QA set targeting critical information.
- Test Cases: A generator model reads your text documents and writes questions focused on critical facts - those whose incorrect answer carries legal, financial, safety, health, security, eligibility, or discrimination / bias risk - each with a ground-truth target answer and supporting quotes.
- Model Response: The evaluated model receives the query, retrieves relevant references from the knowledge base, and produces a response.
- Hallucination Scoring: The model's response is broken down into atomic factual claims. Each claim is first checked against the retrieved references to determine whether it is supported. Claims that are not supported are then checked against the ground-truth target to determine whether they are nevertheless correct.
- Score Computation: The hallucination score is calculated as the fraction of response claims that are both unsupported by the retrieved references and incorrect with respect to the target.
The evaluation focuses on claim-level hallucinations. A higher score indicates that a larger fraction of the model's response consists of claims that are neither grounded in the retrieved references nor correct according to the target answer.
Claim extraction, support checking, and correctness checking are performed by a judge model, which is configurable through the task configuration.
Scoring
Hallucination Scorer
Examples
Faithful response - not hallucinated
- references:- Refund requests for cancelled flights must be submitted within 90 days of the cancellation date.
The claim is supported by the retrieved references and correct with respect to the target.
Hallucinated response - unsupported and incorrect
- references:- Refund requests for cancelled flights must be submitted within 90 days of the cancellation date.
The claim (30 days) is neither supported by the retrieved references (which state 90 days) nor correct with respect to the target. On a critical, deadline-bearing question this fabrication could cause a user to forfeit a refund.
References
[1] Ru D, Qiu L, Hu X, Zhang T, Shi P, Chang S, Jiayang C, Wang C, Sun S, Li H, Zhang Z. Ragchecker: A fine-grained framework for diagnosing retrieval-augmented generation. Advances in Neural Information Processing Systems. 2024 Dec 16;37:21999-2027.