Retrieval-augmented generation (RAG) lets an AI application answer questions using selected documents or other knowledge sources. It searches those sources for relevant passages, then gives the passages and the user’s question to a language model to generate an answer. This can ground responses in material that changes without retraining the model for every update—but retrieval can miss important information, and supplied context does not guarantee a correct answer.
What is RAG?
RAG combines information retrieval with language-model generation. Instead of relying only on what a model learned during training, an application searches an external or private knowledge source when a question arrives. It passes the most relevant results to the model as context, along with the question.
As an Amazon Associate I earn from qualifying purchases.
The model then generates a response conditioned on that context. A RAG system may also preserve links from retrieved passages to their original documents so it can show where an answer came from. RAG is an application architecture, not a single model or database; its components and behavior depend on how the system is assembled. AWS describes the main RAG components and flow.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchHow does a RAG system work?
There are two related flows: preparing the knowledge before questions arrive, and retrieving information each time someone asks a question.
1. Prepare and index the knowledge
- Connect to the source. Collect documents or other supported material from the locations the application is allowed to use.
- Extract and process content. Convert source files into text or another searchable representation. Extraction quality matters: missing tables, headings, or other content can leave the index incomplete.
- Split content into chunks. Divide material into passages that can be searched and supplied as context. Preserve useful structure, such as section boundaries, rather than assuming one chunk size works for every source.
- Add metadata where useful. Information such as document identity, section, date, or access permissions can help filter results and retain provenance.
- Create embeddings and store searchable records. An embedding model converts text into numerical representations for semantic search. The index may also contain the chunk text and metadata; preserve its mapping to the original source if users need citations or verification.
2. Retrieve and generate at question time
- Receive the question. The application accepts the user’s query and determines what search or access rules apply.
- Find candidate passages. A retriever searches the index, using keyword search, vector search, or a combination, and returns likely relevant chunks.
- Assemble context. The application selects and formats the retrieved material for the model. Ranking can help put the most useful passages first.
- Generate the response. The model receives the question and retrieved context and produces an answer. The application can show source references alongside it.
Ingestion is generally done when material is first added or updated; retrieval and generation happen again for each query. AWS documents this prepare, retrieve, and generate pattern for its knowledge-base service.
What components does a RAG application need?
A vector database is only one possible part of the system. A working application also needs a way to connect to sources, process their contents, retrieve and rank passages, call a model, and manage the full request.
| Component | What it does |
|---|---|
| Connectors and ingestion | Collect source material and handle changes or new documents. |
| Extraction and chunking | Turn source content into passages suitable for indexing and retrieval. |
| Embedding model and index | Represent content for semantic search and store searchable records. An index can also support full-text search. |
| Retriever and ranker | Find candidate passages and order or refine them for relevance. |
| Orchestrator | Coordinate retrieval, context assembly, model calls, and any other application steps. |
| Language model | Generate a response based on the question and the context it receives. |
| Guardrails and user experience | Apply application controls and present answers, sources, and feedback mechanisms. |
Some managed services combine several of these responsibilities; a custom pipeline can give a team more control but requires it to operate the pieces. The right division depends on the source formats, access rules, retrieval needs, and operational capacity. AWS outlines these production components.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Do you need a vector database for RAG?
No. RAG requires a way to search the knowledge source and supply relevant results to the model; it does not require one specific database type. Full-text search can be effective when exact words, names, or identifiers matter. Vector search can find semantically related passages even when the wording differs. Hybrid search combines lexical and vector methods, which can help when both exact terminology and meaning matter.
Choose based on the questions people ask and the language in the source material. Microsoft recommends evaluating search options rather than assuming vector search alone is sufficient. Its overview explains full-text, vector, and hybrid retrieval techniques.
How should you choose chunks and retrieval techniques?
Chunk around meaning and source structure
A chunk should contain enough information to answer a likely question while remaining focused enough to retrieve usefully. A heading separated from its explanation, for example, may be less helpful than a passage that keeps them together. Available approaches include sentence-based, fixed-size, custom, layout-analysis, and machine-learning-assisted chunking. Metadata can support filtering or improve search. Test alternatives using representative documents and realistic questions instead of treating any one chunk size as universally best. Microsoft’s RAG techniques guide discusses chunking approaches and metadata.
Add rewriting or reranking only when it helps
Query rewriting creates alternative versions of a question before search; it may help when users phrase questions ambiguously or informally. Reranking takes an initial set of candidates, scores them again for relevance, and forwards a smaller set to the model. Both add processing and complexity, so compare results with and without them on the application’s test questions before adopting them.
Start with a fixed retrieval flow; use agentic retrieval for harder tasks
A standard RAG flow follows a defined sequence: search, assemble context, and call the model. It is a useful baseline when a question can be handled by one search against one index. Agentic retrieval lets an agent decide when and how to search, potentially decomposing a question into subqueries or selecting among sources. That flexibility adds orchestration and should be justified by a need for multistep reasoning or dynamic source selection. Microsoft’s recommendation to consider agentic retrieval for new implementations is specific to its Azure AI Search context, not a universal requirement for every RAG system. Microsoft explains standard and agentic retrieval in its Azure AI Search documentation.
How do you build and evaluate a RAG system?
Begin with a defined use case, representative source material, and realistic questions—including questions the source collection cannot answer. Evaluate the pipeline in stages so a weak answer does not automatically lead to changing the language model.
- Check extraction and chunking. Confirm that important source content is present in the index and divided into useful passages.
- Test retrieval independently. For each question, inspect whether the relevant information appears among the retrieved results. Compare search methods and configuration, then check whether ranking puts useful passages near the top.
- Inspect context assembly. Verify that the passages actually sent to the model are relevant, sufficient, and not displaced by less useful material.
- Evaluate generated answers. Check whether responses are grounded in the supplied passages, complete for the question, relevant, and making effective use of the retrieved context. Microsoft names groundedness, completeness, utilization, and relevancy as possible response-evaluation metrics; suitable criteria and thresholds depend on the application.
- Track configurations and aggregate results. Test across a set of questions and record the pipeline settings so comparisons reflect overall behavior, not a single example.
If answers fail, trace the failure through source coverage or freshness, extraction, chunking, embeddings, search configuration, ranking, context selection, and prompt assembly. Microsoft’s design and evaluation guide describes these stages and evaluation considerations.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How do you protect private data in a RAG system?
Retrieval can expose source material to people who should not see it if access rules are absent or applied inconsistently. Enforce authorization throughout ingestion and retrieval, and use metadata filters so a query returns only content the user is permitted to access. Consider redaction at multiple stages, and secure storage and inference as well as the prompt itself.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsKeep source mappings where users need to verify answers, but ensure that showing a citation does not reveal a document the user cannot access. Guardrails can help address accuracy, responsible use, ethics, hallucinations, and bias; they do not guarantee that errors or unsafe outputs will be eliminated. AWS security guidance discusses controls for access to data and systems in generative AI applications.
Should you use a managed service or build a custom pipeline?
A custom pipeline can offer control over ingestion, chunking, search, ranking, and orchestration, but the team must integrate and operate those components. Managed services can reduce the amount of infrastructure a team assembles; their fit depends on the sources and formats they support, retrieval controls, access-control integration, citations and observability, and evaluation workflow.
Amazon Bedrock Knowledge Bases and Azure AI Search are examples of provider-documented managed options, not a neutral ranking of platforms. Compare them against the actual requirements above and verify current capabilities with each provider before choosing. The available documentation does not establish a neutral current cost comparison or a universally best provider. Amazon Bedrock Knowledge Bases documentation and Azure AI Search documentation describe their respective approaches.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
