October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PCOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk7 min

RAG Architecture Diagram: What Each Box Does and What Each Arrow Costs

A practical guide to the two RAG flows—ingestion and serving—with each arrow’s payload, cadence, cost drivers, latency trade-offs, and implementation options.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

A retrieval-augmented generation (RAG) system has two connected flows: an ingestion path that prepares and indexes content when it is added or changed, and a serving path that repeats retrieval and model work for user requests. The arrows carry different data and incur different work—such as document processing, embedding, search, and generation—so there is no meaningful universal price per arrow without specifying the provider, region, workload, and configuration.

How to read a RAG architecture diagram

Think of the diagram as two flows meeting at the index. Ingestion turns source material into searchable chunks and vectors. Serving turns a user’s question into a search, selects evidence, and gives that evidence to a language model along with the question.

As an Amazon Associate I earn from qualifying purchases.

Ingestion and updates:
Source systems → connector or file landing → parse, clean, and chunk → document embedding model → vector index or store

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Repeated request and response:
User → UI or API → orchestrator → query embedding → retrieval → optional hybrid merge or reranking → prompt assembly → LLM inference → optional safety checks → answer with supporting sources → observability and feedback

#1 Best Overall
VSXLEOZ Vintage History of Architecture Poster Knowledge Canvas Wall Art Aesthetic Decorative Painting Living Roomstylestyle 12x18inch(30x45cm)
  • 👑Poster gets 0.6-2,4cm more widely incase to protection.The new frameless wall art poster print is made of durable, hardwearing,dust and ash resistant canvas to ensure the authentic.
  • 👑This poster extraordinary wall decoration will give your room a new look. It is very suitable as a Christmas or birthday gift to family and friends. Add more color to your bedroom with these beautiful wall decorations while showcasing your favorite artists.
  • 👑 Poster wall display aesthetics can be used in many ways - the traditional way is to stick a poster to your wall in any pattern.Alternatively, you can hang them from cloth pins on the bed. You can also try attaching it to the wall with a frame of the corresponding size
  • 👑A perfect wall decoration painting adds an elegant artistic atmosphere to your home, living room, bedroom, kitchen, apartment,office, hotel, restaurant, office, bathroom, bar, etc. Suitable for all modern graphic and photographic designs.
  • 👑If you are not satisfied with our poster print paintings, please feel free to contact us. We will do our best to provide you with thebest shopping experience.

These are logical responsibilities, not a requirement to deploy one product for every box. A managed platform may combine several steps; a self-managed design may expose them as separate services. Provider reference architectures are examples of implementation, not universal blueprints.

What the ingestion arrows carry and cost

Ingestion work is usually triggered by initial loading, scheduled refreshes, or detected changes. A large backfill can create a burst of processing and indexing even if normal updates are infrequent.

Rank #2
Pop Chart | Architecture of American Houses | 16" x 20" Art Poster | Complete History of American Homes | Thoughtful Housewarming Gift and Wall Decor | 100% Made in the USA
  • Home in on the History of US Housing Architecture: Whether you're an architecture buff or lover of all things Americana, this groundbreaking survey of American house styles is perfect for placing on the wall of your own cherished nest.
  • A Detailed Diagram of Domiciles: From 17th century Postmedieval English abodes to 19th century Tudors all the way through the “McMansions” of the 1990s, this breakdown brings together 121 American houses in all--sorted into seven major categories and 40 subdivisions.
  • Premium Printing: Printed in the United States via offset lithographic process onto durable, acid-free 100-lb cover stock paper, this museum-quality print will elevate your wall decor for decades to come.
  • Ready for Your Walls: Each print ships in sturdy, premium packaging that is suitable for gifting as is! (Prefer to frame it first? Measuring 16" x 20,” this standard-sized print is simple to find a frame for).
  • From the Infographic Masters at Pop Chart: Established in 2010, our studio has created hundreds of eye-catching art prints on every subject you can imagine—from national parks to space travel to sports!
Arrow Payload and operation Cadence, cost, and latency dimension
Source → connector or landing zone Files, records, or change events are brought into the RAG system. Runs on arrival, refresh, or backfill. Account for connector development and operation, source licensing where applicable, transfer, and landing-zone storage. Source formats and available connectors vary.
Landing zone → parser and chunker Raw content is fetched, extracted, cleaned, normalized, and split into retrievable chunks. Runs for new or changed material. Processing time, retries, and transient storage contribute to cost; scanned or complex-layout documents may also need OCR or layout extraction. Processing duration depends on the corpus and extraction method.
Chunks → document embedding model Each chunk is converted into a vector representation. Typically incurred for initial ingestion and changed chunks, not every user request. Embedding inference or compute grows with the amount of text processed; chunk size and overlap affect the number of vectors. In Google’s reference design, indexed content and runtime queries use the same embedding model and parameters.
Vectors and associated text or metadata → index or store Vectors, searchable content, and metadata are written, indexed, and kept available for retrieval. Index build and update work occurs as content changes; storage and serving capacity continue while the index is retained and available. Managed vector search and database-backed vector search allocate operational work and capacity differently, so the box does not imply a single billing model.

What the serving arrows carry and cost

Serving is the repeated path. Some steps can be combined or skipped, but a typical request may involve an embedding call, retrieval, and generation—in addition to application work.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Arrow Payload and operation Cadence, cost, and latency dimension
User → UI, API, and orchestrator A natural-language question and, where relevant, conversation context enter the application. The orchestrator coordinates downstream work. Occurs per request. Include application compute, authentication, networking, request logging, and any session or state storage. This work may be smaller than model inference, but it should still be measured. The placement of the orchestrator varies by design.
Question → query embedding The request is encoded as a vector compatible with the indexed vectors. Usually occurs for each search request. Include query-embedding inference or compute and a serial network or API hop. Google’s design uses the same embedding model and parameters for indexed data and runtime queries.
Query vector → retriever and index The retriever searches for candidate chunks and returns their text or metadata. Search may be vector-based or combine vector and lexical results. Occurs per retrieval. Cost and response time depend on search requests, index or database serving capacity, filters, and result transfer. Asking for more candidates can improve recall but sends more material to later steps.
Candidates → hybrid merge or reranker (optional) Ranked results from multiple retrieval methods may be merged; a reranker can score the query and candidates together and reorder them. Optional work on each request that uses it. It adds compute or model calls and a serial stage. Microsoft’s Azure guidance notes that reranking adds latency compared with standard, vector, or hybrid searches. Benchmark whether its relevance improvement justifies the added time and cost.
Retrieved context → prompt assembly The original question, instructions, and selected evidence are formatted into the model’s input. Occurs per generation request. Orchestration itself uses compute, but the main downstream consequence is prompt size: more or longer chunks generally mean more model input tokens. Redundant context can add work without helping the answer, so tune context selection and answer quality together.
Prompt → generator or LLM The model receives instructions, the question, and retrieved context, then generates an answer. Occurs for each answer generated. Model input and output inference or compute, serving capacity, time to first token, and completion latency all matter. Input context and output length affect work. A portable dollar figure cannot be inferred from the diagram alone.
Model → safety or response processing → user Generated text may be screened or filtered, formatted with citations, and returned to the user. Per request when configured. Include safety-service calls or compute and response transport. Safety may be a separate hop or part of the model platform; Google’s managed architecture includes configured safety filters.
Request and answer → logs, metrics, and evaluation Operational events and, where appropriate, sampled prompts and responses feed monitoring and quality analysis. Generated alongside requests or processed later. Account for log volume, retention, analytics, evaluation compute or model calls, and data governance. This work is easy to omit from a small diagram, but production monitoring and quality evaluation require it.

Optional boxes belong only where they solve a measured problem

  • Query rewriting, augmentation, or decomposition: A preparatory step can clarify a vague question, add useful search terms, or split a multi-part request. It can also add model work and a serial call. Use it for query types where evaluation shows a benefit.
  • Hybrid search: Combine lexical and vector retrieval when exact names, identifiers, or terms matter alongside semantic similarity. Because it can produce multiple ranked lists, the diagram may need a merge step.
  • Reranking: Consider it after broad retrieval when ordering candidates more accurately is worth additional latency and compute. Compare relevance and latency on representative queries rather than treating reranking as a free default.
  • Graph traversal: Add this when entity relationships and multi-hop connections are central to the task. It is an advanced retrieval choice, not a required RAG component.
  • Evaluation loop: Draw an offline feedback arrow from logs or test sets back to chunking, retrieval, prompts, and model configuration. Evaluation can score dimensions such as factual accuracy and relevance.

How to estimate the cost of a RAG design

First separate work that happens when content is ingested from work repeated when users ask questions. Then identify which parts are billed per operation and which require capacity that stays provisioned. Report recurring request costs separately from fixed or provisioned capacity when the chosen service bills them differently.

Rank #3
AI Architecture Blueprint Poster - Backend Endpoint Map - 13x19
  • DETAILED AI BLUEPRINT DESIGN: Features a comprehensive diagram of the AI Backend Endpoint Map, showcasing intricate connections across Authentication, Inference, Training, and Monitoring sections.
  • GLOSSY PRINT QUALITY: Printed on high-quality glossy paper that delivers vibrant deep blues and crisp whites for excellent readability and a polished, professional appearance.
  • IDEAL SIZE FOR ANY SPACE: Measuring 13x19 inches in portrait orientation, this unframed poster fits perfectly in offices, studios, hallways, and tech-themed rooms.
  • DUAL PURPOSE DECOR: Serves as both a sophisticated wall art piece and a handy mini reference guide for AI architecture concepts, making it great for professionals and enthusiasts alike.
  • PERFECT GIFT FOR TECH LOVERS: An excellent choice for anyone passionate about AI and technology, ideal for decorating educational spaces, modern offices, or creative studios.

A useful estimate needs assumptions for:

  • Ingestion volume and update rate, including the size of any initial backfill.
  • Parsing and extraction needs, chunk size, overlap, and resulting chunk count.
  • Embedding model, vector count and dimensions, and index or replica configuration.
  • Request rate, retrieval method, filters, and number of candidate results.
  • Whether query rewriting or reranking is enabled.
  • Prompt context size and expected generated output length.
  • Provider, region, capacity mode, and serving configuration.
  • Logging, evaluation, and retention volume and duration.

Without those assumptions, an arrow-by-arrow dollar total would hide more than it explains. Architecture references establish the steps and trade-offs, but do not by themselves define a comparable bill for a fixed workload.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Where the boxes can live

The vector index is a function in the design, not necessarily a standalone vector-database product. Hosting and storage choices change who operates components and how capacity is managed.

Rank #4
Dazoratix Travel City Wall Art - 9 Pcs Vintage Cityscape Prints Decor Poster Famous Architecture Landscape Artwork Buildings Aesthetic Artcat Pictures Paintings for Living Room Bedroom Home (Unframed)
  • Wall Art Prints: This city wall art decor set includes 9 unframed posters, each measuring 10 × 8 inches, making them easy to arrange together or display separately. The compact size allows these cityscape prints to fit into various spaces such as living rooms, bedrooms, dorms, or offices, adding personality and color to your walls
  • Famous Cityscape Design: Each artwork features iconic landmarks including New York, San Francisco, Las Vegas, Tokyo, Barcelona, Mexico City, Dublin, Stonehenge, and Cairo. The unique cityscape illustrations bring a blend of culture, history, and travel inspiration, making these city wall art prints a beautiful choice for people who love world architecture and aesthetic wall decor
  • High Quality Material: Printed on premium cardstock with reliable ink technology, these city wall posters are fade resistant and maintain vibrant colors over time. The smooth surface and clear details ensure that every building and landscape is displayed in artistic quality, creating a stylish upgrade for your wall decoration
  • Easy to Use: These unframed city wall art prints are simple to hang or frame. You can place them directly on the wall with tape, clips, or pins, or insert them into standard frames for a more polished look. Their lightweight design makes it easy to change the arrangement anytime to match different moods or occasions
  • Ideal Home Decoration: This cityscape wall decor set is suitable for decorating the living room, bedroom, study, hallway, or even a creative office space. It also makes a thoughtful gift for students, travelers, art lovers, or anyone who enjoys home decoration. These aesthetic wall prints bring charm, culture, and inspiration into any environment
Pattern What it illustrates Trade-off to assess
Managed vector search and hosted models Google Cloud’s reference architecture uses managed services for ingestion, embeddings, vector search, and generation. It can reduce self-operated infrastructure work, while choices remain tied to service configuration and region.
Self-managed serving with database-backed vectors Google’s GKE example places frontend, inference, and embedding services in GKE and stores vectors in PostgreSQL with pgvector. It offers a concrete route for teams choosing open models and more infrastructure control, with corresponding infrastructure operations to own.
Managed platform with a relational vector store Google’s AlloyDB reference design uses a PostgreSQL-compatible vector store and separates ingestion, serving, and quality evaluation. It demonstrates that vector retrieval can be provided by a relational database rather than a standalone vector product.

Compare designs on operational ownership, scaling and capacity model, data locality and access control, retrieval quality on the actual corpus, request latency, model choice, observability, and total cost for the measured workload. “Vector database versus no vector database” is often the wrong framing: the relevant question is which component provides vector retrieval and whether it fits the workload.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Windows Errors? Fix Them Before They SpreadFree repair scan
Outdated Drivers Are Slowing You DownFree scan - exact matches

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.