---
title: "How to Tell Whether a RAG Failure Came from Retrieval or Generation"
description: "Use staged checks to find whether the right evidence was missing, buried, ignored, or misrepresented."
canonical: "https://www.evalgate.com/guides/diagnose-rag-retrieval-vs-generation"
date-published: "2026-09-02"
last-updated: "2026-09-02"
category: "RAG and hallucinations"
keywords: "RAG retrieval failure, RAG generation failure, debug RAG pipeline"
---

# How to Tell Whether a RAG Failure Came from Retrieval or Generation

Use staged checks to find whether the right evidence was missing, buried, ignored, or misrepresented.

**Question:** Was my RAG failure caused by retrieval or by the LLM?

## Short answer

Check the pipeline in order. If the required evidence was not retrieved, diagnose indexing, query transformation, filters, or ranking. If adequate evidence was present but the answer ignored, contradicted, or overstated it, diagnose generation and prompting.

## Key takeaways

- Never debug the final answer without the retrieved passages.
- Test retrieval sufficiency before grading answer faithfulness.
- Preserve hybrid failures where weak evidence and poor generation interact.

## Signs this is the problem

- Teams rewrite the prompt when the needed document never appeared.
- Top-k contains the answer but ranking buries it below distracting context.
- The generator invents specifics despite sufficient authoritative evidence.

## Step-by-step approach

### 1. Define required evidence

Mark the document or passage needed to answer each test question correctly.

### 2. Evaluate retrieval

Measure whether required evidence appears, where it ranks, and whether filters or freshness removed it.

### 3. Evaluate generation

Given the captured context, check answer correctness, faithfulness, completeness, and appropriate abstention.

### 4. Fix the first broken stage

Change indexing, ranking, context assembly, or generation independently and rerun both layers.

## What to measure

| Metric | What it measures | How to use it |
| --- | --- | --- |
| Evidence recall | Cases where required evidence appears within the usable context window. | Fix retrieval when this misses regardless of answer quality. |
| Context precision | Retrieved content that is relevant enough to support the answer. | Reduce distractors when the evidence is present but diluted. |
| Conditional faithfulness | Answer grounding among cases with sufficient retrieved evidence. | Use this to isolate generation behavior. |

## Common mistakes

- Using answer correctness as a proxy for retrieval quality.
- Changing chunking and prompts in the same experiment.
- Forcing an answer when the correct result is insufficient evidence.

## Practical checklist

- [ ] Define required evidence: Mark the document or passage needed to answer each test question correctly.
- [ ] Evaluate retrieval: Measure whether required evidence appears, where it ranks, and whether filters or freshness removed it.
- [ ] Evaluate generation: Given the captured context, check answer correctness, faithfulness, completeness, and appropriate abstention.
- [ ] Fix the first broken stage: Change indexing, ranking, context assembly, or generation independently and rerun both layers.

## Where EvalGate fits

EvalGate's trace and evaluation evidence can score retrieval and generation separately while preserving their shared run context.

[Evaluate RAG systems](https://www.evalgate.com/docs/guides/rag-evaluation) or [start with the EvalGate quickstart](https://www.evalgate.com/docs/quickstart).

## Frequently asked questions

### What should I do first?

Mark the document or passage needed to answer each test question correctly

### How should the result be measured?

Cases where required evidence appears within the usable context window. Fix retrieval when this misses regardless of answer quality.

### When is the change ready to ship?

Ship only after the protected cases pass, the primary metric clears its agreed boundary, and the team reviews the remaining failure modes instead of relying on one aggregate score.

## Related guides

- [https://www.evalgate.com/guides/measure-rag-faithfulness](https://www.evalgate.com/guides/measure-rag-faithfulness)
- [https://www.evalgate.com/guides/test-rag-chunking-changes](https://www.evalgate.com/guides/test-rag-chunking-changes)
- [https://www.evalgate.com/guides/build-rag-golden-dataset](https://www.evalgate.com/guides/build-rag-golden-dataset)