Because a fact being present in the source file does not mean the system extracted it, retrieved it, included it in the model’s prompt, or gave the model enough evidence to answer. Retrieval-augmented generation (RAG) is a chain of stages, and a miss can happen at any one of them. The fastest way to fix it is to trace one failed question through that chain and identify the first point where its supporting evidence disappears or becomes unusable.
How information gets lost in a RAG pipeline
A typical RAG system processes a question and documents through ingestion, indexing, retrieval, context assembly and generation. Each stage transforms or filters evidence. The NVIDIA RAG pipeline overview describes these stages, while GOV.UK’s RAG workflow explains how preprocessing and context consolidation fit into the process.
As an Amazon Associate I earn from qualifying purchases.
That means “it is in the document” is only the starting point. The passage must survive each step and reach the model in a form that supports a definitive answer.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Find the first stage where the evidence fails
Use a question with a known answer and a specific supporting passage. Follow that evidence forward through the system, rather than changing retrieval settings at random.
#1 Best Overall
1. Ingestion and extraction
Confirm that the expected file and version entered the system, then inspect the extracted text. A PDF or image can look correct to a person while its text extraction omits a scanned page, table, header or layout-dependent relationship. Preprocessing needs to fit the data type; GOV.UK’s workflow discusses separate handling for PDFs and images.
2. Chunking and indexing
Inspect the actual indexed chunks around the answer, not just the original document. A chunking strategy can separate a fact from its heading, unit, exception or antecedent. The resulting text may be hard to match or misleading on its own. Chunk size and splitting strategy affect context relevance, and changes to chunking or embeddings may require re-indexing. See GOV.UK’s workflow and Databricks’ quality overview.
3. Query and embedding alignment
Check that the query and document chunks receive compatible preprocessing and that retrieval uses the embedding model associated with the indexed chunks. Differences in cleaning or embedding models can make relevant text less findable. Microsoft Learn’s information-retrieval guidance recommends consistent cleaning and use of the model that embedded the chunks.
4. Candidate retrieval and filters
Record the exact query, collection or index, filters, candidate passages, scores, ranks and top-k limit. A wrong collection, overly restrictive filter or shallow candidate set can exclude the passage before later stages see it. Semantic similarity can also miss literal terms such as names, codes or distinctive phrases; full-text and vector search address different retrieval needs, and hybrid search can combine them. Microsoft documents those approaches, along with filters and decomposed subqueries, in its retrieval guide. For tracing configuration and retrieval behavior, see NVIDIA’s debugging guide.
5. Reranking and context assembly
Compare the initial retrieval results with the reranker’s output and with the exact context passed to the model. A relevant result can be demoted, dropped, or shortened while the system assembles context to fit the model’s token limit. NVIDIA distinguishes retrieval and optional reranking in its pipeline description; GOV.UK describes consolidating context for token limits in its RAG workflow.
6. Generation and evidence sufficiency
If the supporting passage is in the prompt, check whether it contains every detail needed to answer and whether other passages conflict with it. A passage can be relevant to the topic but still omit the decisive fact. Google Research defines sufficient context as context containing all information needed for a definitive answer; it treats incomplete, inconclusive or contradictory context as insufficient. Its 2025 explanation of context sufficiency is useful for separating a retrieval hit from evidence that actually answers the question.
A practical debugging sequence
- Choose one failed question. Write down its expected answer and the passage that supports it. Preserve the original question and capture any rewritten or expanded version the system uses.
- Verify the source and index. Confirm the correct file version and collection, then inspect extracted text and the indexed chunks around the supporting passage.
- Trace retrieval. Log the query, filters, candidate IDs, scores and ranks. If it is safe and consistent with access controls, compare with a retrieval run that removes suspected narrowing filters.
- Trace ranking and prompt assembly. Compare retrieved candidates with reranker results, then inspect the exact context supplied to the language model.
- Classify the failure. If the passage is absent from the index, investigate extraction or indexing. If it is indexed but absent from candidates, investigate query alignment, filters and retrieval. If retrieved but missing from the prompt, investigate ranking or context assembly. If present but insufficient or ignored, investigate evidence coverage and generation.
- Change one variable at a time. Re-run the same failed-question set and measure retrieval and answer quality separately. Databricks’ quality overview treats retrieval quality and generation quality as distinct, interacting concerns.
Choose a fix that matches the failure
| Observed failure | Intervention to test | What to compare |
|---|---|---|
| Text is missing or garbled after extraction | Correct document preprocessing or extraction; verify the extracted passage before re-indexing. | Whether the target text and its relevant table or layout relationships are preserved. |
| Indexed chunk loses heading, unit or exception | Adjust chunk boundaries or preserve useful section metadata; re-index if required. | Recall of known supporting passages and whether chunks remain understandable on their own. |
| Query wording does not match indexed content | Align preprocessing and embedding models; test full-text or hybrid retrieval for literal terms. | Passage recall, irrelevant-result noise and answer correctness. |
| A filter, collection or shallow candidate set excludes the passage | Correct the collection or filter; test a deeper candidate set where appropriate. | Whether the passage enters the candidate list, alongside latency and noise. |
| Passage is retrieved but omitted or demoted later | Inspect reranking and context assembly; test ranking or consolidation changes. | Whether the needed evidence survives into the final prompt and supports the answer. |
| Passage is in the prompt but does not settle the question | Check for missing details, conflicting evidence or a multi-part question; test query decomposition or clearer answer instructions. | Whether the assembled evidence is sufficient and whether answers are correct. |
Query rewriting, augmentation, decomposition and hypothetical-document methods can help when wording or multi-part questions are the problem. They can also change the question’s intent, so inspect the transformed query and confirm it still asks for the original information. Microsoft describes these as optional query-translation methods and cautions that augmentation should preserve the query’s nature in its retrieval guidance.
Do not treat a larger top-k as a universal repair. More candidates can increase noise and affect latency, while an extraction failure or exclusionary filter will remain. Evaluate candidate recall, context precision, answer correctness, latency, compute or storage cost, implementation complexity and whether re-indexing is needed; no single configuration is established as best for every system. The relevant design trade-offs are covered in the Microsoft retrieval guide, NVIDIA pipeline documentation and Databricks quality overview.
What context sufficiency does—and does not—tell you
Google Research reported that an optimized prompted-LLM method classified sufficient-context examples with at least 93% accuracy. That figure is the accuracy of a method for labeling context sufficiency, not the accuracy of RAG answers. The authors also report that their human evaluation set contained 115 question-and-context examples. See the Google Research article published May 14, 2025.
Keep security boundaries intact while debugging
Retrieved passages are data, not trusted instructions. Do not disable access-control filters as a casual retrieval experiment: permissions are a security boundary, not just a relevance setting. Preserve access-control metadata through chunking and enforce permissions at retrieval time. The OWASP RAG Security Cheat Sheet also discusses attacks carried through context windows.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
