Make RAG accuracy a pipeline discipline, not a single score. Test whether retrieval finds the right evidence, whether the model stays within that evidence, whether it answers every part of the request, and whether the evidence itself is correct. A system can be perfectly grounded yet incomplete, or retrieve excellent passages and still produce an irrelevant answer. Separating those failure modes gives your team specific fixes instead of an unhelpful “accuracy” percentage.
What “accurate” means in a RAG application
Retrieval-augmented generation (RAG) searches a source corpus, places selected passages in a model prompt, and asks the model to answer using that context. Accuracy therefore spans at least two stages: finding useful evidence and generating a defensible answer from it. Microsoft’s RAG evaluator guidance treats retrieval as a process to assess separately from the final response, while its end-to-end evaluation guidance recommends metrics suited to the actual workload rather than a universal threshold (Microsoft Foundry RAG evaluators; Microsoft Azure Architecture Center evaluation phase).
As an Amazon Associate I earn from qualifying purchases.
Define success in terms of the task your users perform. A policy assistant may need to identify the current rule, quote the qualifying exception, and tell the employee what to do. A support assistant may need to diagnose a product issue and cite the applicable procedure. Those are different workloads and require different test questions and acceptance criteria.
- Retrieval quality: Did the search return the passages that contain the answer?
- Groundedness (faithfulness): Are the response’s claims supported by the supplied context?
- Completeness: Did the response cover all material parts of the question?
- Relevance: Did it address the user’s actual request without digressions?
- Correctness: Are the source facts and the resulting statements factually right?
- Utilization: Did the model use the available context appropriately rather than ignoring or misapplying it?
These dimensions are complementary. Groundedness resembles precision—avoiding unsupported claims—while completeness resembles recall—covering the expected information. Neither alone proves that an answer is useful or true.
#1 Best Overall
Build the evaluation foundation before tuning prompts
Evaluation is only meaningful when the test environment resembles production. Assemble the data and retrieval components first, then measure the complete workflow. Microsoft’s architecture sequence is to gather documents and queries, chunk and enrich the chunks, create embeddings, build a search index, implement a search strategy, and then evaluate end to end (Azure Architecture Center).
Prepare a representative corpus
Include the document versions and access rules your application will actually use. Record source identifiers, publication or revision dates, ownership, and document type. A response cannot be more current than the corpus it searches, so stale or contradictory documents are a correctness problem even when the model quotes them faithfully.
Create a workload-shaped question set
Build a stable set of questions from real support tickets, analyst requests, policy lookups, and product workflows. Include common questions, complex questions, multipart requests, ambiguous wording, misspellings, and questions whose answer is “the corpus does not establish this” when those cases occur in production. For each item, record the expected critical facts, acceptable sources, and any required caveat. Google Cloud recommends “golden questions” that represent intended use cases and incremental testing (Google Cloud evaluation guidance).
Rank #2
Label retrieval ground truth where possible
For each question, identify the documents or chunks that are relevant. Microsoft notes that document-retrieval evaluation requires retrieval ground truth (Microsoft Foundry). Labels need not prescribe one exact passage: mark all passages that contain necessary evidence, plus passages that are clearly irrelevant. If experts disagree, preserve that disagreement and define the condition under which each answer is acceptable.
Measure retrieval before blaming the language model
Run the retriever alone against the labeled questions. Inspect whether relevant evidence appears in the returned set and where it ranks. If the answer-bearing passage never arrives, generation cannot reliably recover it; changing the prompt or model only masks the underlying defect.
Diagnose missing or noisy context
- Corpus coverage: Add missing policies, product versions, regional documents, or permissions-scoped records.
- Chunking: Revisit chunk size and overlap so definitions, tables, exceptions, and procedural steps stay together without burying the key sentence in excessive text.
- Enrichment: Add titles, section paths, dates, product identifiers, and synonyms that help lexical and semantic search distinguish similar passages.
- Embeddings and vocabulary: Check whether domain acronyms, codes, names, and multilingual terms are represented by the chosen embedding model.
- Search strategy: Test hybrid lexical-plus-vector retrieval, metadata filters, query rewriting, and reranking where the workload requires them.
- Retrieval depth: Increase or decrease the number of candidates deliberately; more chunks can improve recall but may dilute the context and increase latency or prompt limits.
When completeness is weak, Microsoft specifically directs teams to inspect the corpus, chunks, embeddings, and whether relevant chunks were missed (Azure Architecture Center). Keep retrieval experiments separate from generation experiments so you can attribute a change to the component that caused it.
Rank #3
Evaluate the generated answer with complementary checks
Groundedness: can every claim be traced to context?
Groundedness asks whether the answer stays within the retrieved evidence. Microsoft defines groundedness, also called faithfulness, as whether the response is based completely on the context (Azure Architecture Center). Check each material claim, including numbers, dates, conditions, and inferred recommendations. A sentence can be fluent and plausible yet unsupported; it should be removed, qualified, or backed by an additional source.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCompleteness: did it answer every required part?
Break multipart questions into expected claims or actions and mark which were covered. An answer that correctly explains eligibility but omits the deadline is grounded but incomplete. Completeness testing is especially important for long policies, troubleshooting trees, and questions containing “and,” “except,” or multiple entities.
Relevance and utilization: did it use the right evidence for the user’s goal?
Relevance checks whether the response addresses the request rather than repeating background. Utilization checks whether supplied context was applied appropriately. A model may quote a true passage about refunds when the user asked about exchanges, or ignore a qualifying paragraph that changes the recommendation. Review both the question and the retrieved context when either metric falls.
Correctness: are the facts and sources themselves sound?
Grounding proves entailment from retrieved text, not that the text is authoritative, current, or internally consistent. Verify high-impact answers against an approved source owner, revision date, or second system of record. If two documents conflict, the application should expose the conflict or apply an explicit precedence rule rather than silently choosing one.
| Signal | Question it answers | Typical failure | First investigation |
|---|---|---|---|
| Retrieval relevance | Did the search return answer-bearing evidence? | Useful passage absent or ranked too low | Corpus, chunking, embeddings, filters, ranking |
| Groundedness | Are claims entailed by retrieved context? | Invented detail or unsupported inference | Prompt constraints, context quality, claim inspection |
| Completeness | Are all critical parts covered? | One condition or requested item omitted | Question decomposition, retrieval depth, answer checklist |
| Relevance | Does the answer address the request? | Off-topic explanation or wrong task | Intent handling, query rewriting, prompt instructions |
| Correctness | Are source and answer facts actually right? | Outdated or contradictory source | Document governance, version precedence, expert review |
| Utilization | Was the supplied context used appropriately? | Important evidence ignored or misapplied | Context ordering, prompt, model behavior, reranking |
Run an iterative, reproducible evaluation loop
- Freeze a baseline. Save the corpus snapshot, chunking and embedding configuration, index settings, retrieval parameters, model identifier, prompt, and application version.
- Run the same evaluation set. Store retrieved chunks, generated answers, latency, token use, evaluator outputs, and human judgments. Responses can vary between runs; Microsoft states that language-model responses are nondeterministic, so retain multiple runs or report ranges where variability matters (Azure Architecture Center).
- Change one meaningful component. For example, alter chunk overlap, add a metadata filter, change reranking, or revise the answer prompt. Avoid changing retrieval and generation simultaneously when you need attribution.
- Compare distributions and examples. Examine metric trends and the individual failures around them. A small average gain can hide regressions in a safety-critical question class; a single threshold can hide wide run-to-run variation.
- Review failures with domain experts. Classify each failure as missing evidence, bad ranking, unsupported claim, omission, irrelevant answer, or incorrect source. Convert recurring classes into new labels, tests, or engineering work.
- Promote only with a regression check. Re-run the full set after every index, model, prompt, corpus, or policy change, and keep the prior results for comparison.
An LLM-based evaluator is a useful judge, not an oracle. Calibrate it against human-labeled examples, give it explicit criteria and the relevant context, and inspect disagreements. Use ranges and trends appropriate to the task instead of declaring that one run establishes a permanent accuracy percentage.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteMake every answer auditable
Preserve the path from a response claim to the retrieved source. Show a readable citation next to each material claim or answer section, with a document title, stable identifier, revision date, and location such as page, heading, or paragraph. Let users open the exact passage used; a generic link to a large document is not enough for review.
Best Value
Google Cloud’s grounding check illustrates this pattern by returning an overall support score and citations to facts supporting claims. Its documentation says: “Perfect grounding requires that every claim in the answer candidate must be supported by one or more of the given facts” (Google Cloud Check grounding with RAG). The same documentation describes current API limits of up to 200 facts, each up to 10,000 characters, and an answer candidate of up to 4,096 tokens. Those are limits of that documented API, not general limits on RAG architectures.
Citations do not establish that a source is correct or that retrieval found every relevant document. Pair claim-level evidence with source governance, freshness checks, access-control enforcement, and retrieval-completeness tests. Log the query, authorized corpus version, retrieved identifiers, prompt or response configuration, answer, citations, and evaluator results under appropriate privacy controls.
Use a failure-troubleshooting map
| Observed symptom | Likely cause | Action |
|---|---|---|
| Answer says “not found” although the document contains the answer | Chunk missed, poor vocabulary match, restrictive filter, or shallow retrieval | Search the corpus directly, inspect chunk boundaries and metadata, test alternate query terms, then rerun retrieval labels |
| Answer contains confident details absent from context | Unsupported generation or ambiguous prompt | Require evidence for each claim, add abstention behavior, inspect the exact retrieved context, and score groundedness |
| Answer cites true text but omits an exception | Incomplete retrieval or answer planning | Label all critical facts, retrieve the exception chunk, and require a checklist for multipart questions |
| Answer uses an old procedure | Stale or conflicting corpus | Record revision dates, filter or rank current sources, define precedence, and flag unresolved conflicts |
| Metrics swing between identical runs | Model nondeterminism or evaluator variance | Repeat runs, report ranges, inspect examples, and avoid overreacting to a single score |
| Adding more context makes answers worse | Irrelevant passages dilute attention or exceed useful context | Improve reranking and filters, reduce retrieval depth, and compare utilization and groundedness together |
Compare evaluation services on your workload, not on marketing claims
Microsoft Foundry and Google Cloud provide documented approaches for RAG and generative-AI evaluation, but the available material does not establish a universal winner or a head-to-head accuracy result. If you are assessing platforms, run the same corpus and golden questions through each and score the same dimensions.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →| Decision axis | Questions to answer in a proof of concept |
|---|---|
| Retrieval evaluation | Can you supply relevance labels, inspect retrieved chunks, and separate retrieval from generation results? |
| Answer evaluation | Are groundedness, completeness, relevance, utilization, and correctness measurable with your data? |
| Evidence display | Can users and reviewers open claim-level citations and stable source passages? |
| Model and process control | Can you use the models, prompts, judges, and human review workflow your organization requires? |
| Operational fit | What latency, throughput, privacy, access-control, data-residency, and cost results occur under your workload? |
The last row requires your own measurements and governance review; the cited documentation does not provide comparative outcomes for those factors.
Set deployment gates that reflect risk
Do not deploy because an aggregate score crossed an arbitrary line. Define gates by question class and consequence. For a low-risk internal search tool, a reviewer might tolerate occasional omissions with visible citations. For legal, financial, medical, safety, or access-control workflows, require human approval, explicit abstention when evidence is missing, and zero tolerance for particular unsupported claims.
- Block release when a critical question loses its authoritative source or produces an unsupported action.
- Require a citation for every externally verifiable material claim.
- Track retrieval, groundedness, completeness, relevance, correctness, and utilization separately in dashboards.
- Sample production conversations for newly emerging intents, source drift, and access-control failures.
- Version evaluation sets so improvements are not caused by silently removing difficult questions.
The practical standard is not “RAG never hallucinates.” It is a measured, inspectable system that finds appropriate evidence, states only what that evidence supports, covers the user’s request, exposes uncertainty, and improves through repeatable tests.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →
