A production RAG system is two connected pipelines: an ingestion job that keeps a searchable, traceable index current, and an online service that retrieves evidence and generates answers within defined latency, cost, and access-control limits. With Python and PostgreSQL plus pgvector, you can combine semantic vector retrieval with PostgreSQL full-text search, rerank candidates, and return citations—but the right chunking, index, and model settings must be measured against your own documents and questions.
What should you decide before building?
Start with the behavior the service must deliver, not a vector index. Write down which collections it can search, how quickly changed documents must appear, who may see each document, what response latency and model spend are acceptable, and what a useful citation must let a reader inspect.
As an Amazon Associate I earn from qualifying purchases.
Create a versioned evaluation set before tuning retrieval. Include representative questions, the supporting documents or passages, required answer facts, and questions the collection cannot answer. A small initial set is useful for iteration, but no particular count is established as sufficient for every application. Add real misses as they occur and retain the set when you change chunking, prompts, models, or indexes.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minute- Freshness: Decide whether ingestion runs on a schedule, after a source change, or through both mechanisms.
- Authorization: Define tenant and document permissions in the data model and queries. Do not rely on a prompt to enforce access.
- Citations: Decide whether answers need document-level references, section-level references, or links that resolve to source locations.
- Budgets: Set request limits, latency targets, and model-call or token limits that the service can enforce.
How should ingestion and online answering fit together?
Keep ingestion separate from the request path. It can run offline or on a schedule; answering is the latency-sensitive path. That separation lets you reprocess documents deliberately without making every user request wait for parsing or embedding.
#1 Best Overall
| Pipeline | Responsibilities | Output |
|---|---|---|
| Ingestion | Load and normalize sources; preserve useful structure; split into chunks; create embeddings; write or replace indexed records; record versions. | Searchable chunks with stable provenance and metadata. |
| Online answering | Check access; transform the query; run lexical and vector retrieval; fuse and rerank candidates; assemble bounded evidence; generate and return an answer. | An answer with citations that map back to retrieved source material. |
Keep the boundaries between retrieval, reranking, and generation clear enough to evaluate them independently. A weak answer can come from missing evidence, poor ordering, insufficient context, or generation that misuses good context; these are different failures and need different fixes.
How do you build a dependable ingestion job?
Load, normalize, and preserve structure
Extract text while retaining headings, sections, and other structure that helps readers locate evidence. Normalize formatting noise, but avoid flattening useful distinctions. Keep tables and code intact when practical: splitting a row from its labels or a code fragment from its explanation can make an otherwise relevant chunk hard to interpret.
Chunk around meaning, not only character counts
Prefer coherent sections or paragraphs over blind fixed-length cuts. The ProductionAIEngineer tutorial (2026) gives 300–800 tokens per prose chunk and 10–15% overlap as example starting values, not validated universal settings. Test alternatives on your corpus: a short chunk may lose necessary context, while a large one can mix topics and make retrieval less precise.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Keep provenance and make reprocessing safe
Give each source document and chunk stable identifiers. Store useful fields such as source, title, section, and content identity alongside the text. These fields support citations, access checks, change detection, and targeted reindexing. Record which embedding model or version produced each vector so you can identify records that need regeneration after a model change.
Make ingestion idempotent: processing an unchanged source again should not create duplicate chunks. When a document changes, update or replace its prior chunk set in a controlled way. An upsert-style write is one way to express this, provided the document and chunk identity rules are stable. Plan reindexing as an explicit operation rather than silently mixing incompatible embeddings.
How should pgvector store and retrieve your chunks?
Enable pgvector in PostgreSQL and store chunk text, embeddings, and relational metadata together when that meets your operational needs. Choose the embedding column dimensions to match the selected model, and keep the distance measure, query operator, and index operator class aligned. The pgvector project documentation supports multiple distance choices; the correct combination depends on the embedding representation and retrieval design.
Keep a relational path for metadata and authorization. Apply tenant and permission filters inside both the vector and lexical retrieval branches before combining candidates. Filtering only after retrieval risks allowing unauthorized material into an intermediate result set, even if it is later removed from the final prompt. Database policies and query logic should reflect the application’s threat model.
Free tools Windows power users keep installed
One-click scans. No signup required.
Choose exact or approximate vector search by measurement
The pgvector project documentation says, “By default, pgvector performs exact nearest neighbor search, which provides perfect recall.” Exact search is a useful baseline for judging whether an approximate index is missing relevant results. HNSW and IVFFlat can trade recall, latency, memory, index size, and build or update behavior differently; neither is a universal default. Compare them on representative queries and data, including the effect of your metadata filters.
| Choice | What it gives you | What to measure |
|---|---|---|
| Exact vector scan | A recall baseline without approximate nearest-neighbor loss. | Query latency and resource use at the data volume and filter patterns you expect. |
| Approximate HNSW or IVFFlat | A faster-search option whose behavior depends on index configuration and workload. | Recall against exact results, latency, memory and index size, and behavior as records are added or changed. |
Why use hybrid search, and how do you combine results?
Dense vector search is useful when a question paraphrases the source. PostgreSQL full-text search can help when the query contains an exact name, phrase, identifier, or technical term. Hybrid search uses both signals so that semantic similarity does not have to carry every retrieval need.
PostgreSQL’s full-text search has its own parsing, matching, ranking, indexing, and language-configuration behavior. Select a text-search configuration appropriate to the corpus language and verify how it treats domain-specific names and identifiers. Run lexical and vector branches with compatible authorization filters.
The two branches may produce scores on different scales. Reciprocal rank fusion combines candidates using their rank positions rather than assuming their raw scores are directly comparable. The pgvector project documentation also describes using a cross-encoder to combine or rerank results. Avoid inventing a weighted blend of unrelated scores unless you calibrate it against your evaluation set.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →How do you rerank and keep context under control?
Retrieve a broader candidate set than you intend to send to the generator, fuse the retrieval lists, then rerank passages for relevance to the actual question. A reranker changes the order of candidates it receives; it cannot recover evidence that neither retrieval branch found.
The ProductionAIEngineer tutorial (2026) illustrates roughly 20 candidates followed by four passages as one pipeline configuration. Treat those counts as an example, not a target: increasing candidates may improve the chance of finding evidence but can increase latency and model cost, while sending too few final passages can omit needed context. Tune both stages using retrieval and answer evaluations under your service budgets.
How do you generate grounded answers with citations?
Construct the generation input from the question and a deliberately bounded set of reranked evidence. Instruct the model to answer from that evidence, distinguish supported claims from missing information, and abstain or clearly state that the corpus does not establish an answer when evidence is insufficient.
Keep a stable mapping from each citation marker to a chunk, document, and section. Return references that resolve to real source material; do not treat generated citation text as proof that a passage supports a claim. Where the system cannot provide a usable source location, avoid presenting a citation as though it were verified.
Keep generation separate from API handling and retrieval logic. This makes it possible to test whether a failure came from retrieval or from answer construction, and to change one component without obscuring the effect of the change.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How do you prove the system works and catch regressions?
Evaluate retrieval and answers separately. For retrieval, check whether relevant source chunks appear among the candidates, including after fusion and reranking. For answers, check correctness and whether required facts are present, supported by evidence, and cited appropriately. Include unanswerable questions and assess whether the service abstains rather than filling gaps with plausible-sounding claims.
- Compare approximate vector results with exact search to estimate recall loss.
- Record misses from real use and add them to the versioned evaluation set.
- Run the same set after changing chunk boundaries, embeddings, index settings, candidate counts, reranking, or prompts.
- Track retrieval outcomes separately from answer quality so that a prompt change does not conceal a retrieval regression.
- Set thresholds from your application’s needs; the cited tutorial does not establish universal quality cutoffs.
What should you monitor and deploy?
A Python API such as FastAPI can expose the online path, while a separate worker or scheduled job handles ingestion. The tutorial uses Python/FastAPI and a container as an example, not a requirement. FastAPI’s deployment concepts guide covers framework deployment considerations; hosting, database operations, and scaling still depend on your environment.
Operationalize the budgets defined for the service: bound request and model input sizes, use timeouts and controlled retries, monitor latency and errors across retrieval and generation, and handle model-provider or database failures explicitly. Store secrets through the hosting environment’s secret-management facilities rather than in source code. Version prompts and evaluation data, and make health checks distinguish an available API from a healthy dependency chain.
Which implementation choices are workload-dependent?
| Decision | Alternatives | Compare on |
|---|---|---|
| Retrieval signals | Dense vector search alone, or dense plus PostgreSQL full-text search. | Paraphrase retrieval, exact-term coverage, ranking quality, and query cost. |
| Rank combination | Reciprocal rank fusion or a cross-encoder-based approach. | Ranking quality, implementation complexity, latency, and model cost. |
| Embeddings | Hosted API or locally hosted/another provider. | Domain-question quality, privacy, latency, operations, model lifecycle, and total cost. Current provider details are not established here. |
| Deployment | Self-managed PostgreSQL and API or managed database/cloud RAG services. | Operational responsibility, data location, integration, cost, scaling, and portability. Google’s Cloud SQL/Vertex AI lab documents one path, not a universal winner. |
For any choice, preserve a working baseline and change one variable at a time where practical. No single chunk size, embedding provider, HNSW setting, candidate count, or hosting model is established as best for every collection.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
