To build a production-ready internal AI assistant on AWS, connect approved company content to Amazon Bedrock Knowledge Bases, but keep identity-aware authorization, answer policy, monitoring and evaluation in your application design. Bedrock can manage substantial parts of retrieval and response generation; it does not by itself determine which employee may see each document or prove that answers are reliable.
How the architecture works
Retrieval-augmented generation (RAG) retrieves relevant company content at question time and supplies it to a foundation model as context. Amazon Bedrock Knowledge Bases provide managed capabilities for connecting data sources to retrieval workflows. An application can retrieve passages for its own processing or use a retrieve-and-generate flow that returns a natural-language response with source context. See How Amazon Bedrock knowledge bases work and AWS’s RAG overview.
As an Amazon Associate I earn from qualifying purchases.
A production system is more than a vector index and a model. It also needs reliable source ingestion, document preparation and metadata, identity-aware retrieval, application orchestration, response safeguards, traceability and evaluation. A practical request path is:
Recommended Free Tools
- Authenticate: The employee signs in through the organization’s identity provider and uses an internal application.
- Resolve access: Application middleware determines the user’s relevant identity or policy attributes and carries them into the retrieval decision.
- Prepare content: Ingestion validates approved sources, extracts and labels content, and maintains document permissions and lineage.
- Retrieve: The application requests relevant passages while enforcing the employee’s access boundary.
- Generate: The selected foundation model receives the question and authorized context; configured safeguards apply to inputs or outputs as appropriate.
- Respond: The application returns a grounded answer with source references, or indicates that the available context is insufficient.
- Operate and improve: Protected operational and audit events support troubleshooting, while representative tests track retrieval and answer quality over time.
This is a design synthesis, not a single AWS-provided reference implementation.
#1 Best Overall
Choose a Knowledge Base operating model
AWS describes Managed and Customer-managed Knowledge Bases. They differ in who operates the pipeline and how much control the organization has over its components; capabilities are not identical. Compare the current Knowledge Bases feature and connector guidance before choosing, since features and regional support can change.
| Decision area | Managed Knowledge Base | Customer-managed Knowledge Base |
|---|---|---|
| Ingestion, indexing, storage and retrieval infrastructure | AWS manages the underlying infrastructure. | Your organization manages the RAG pipeline and vector store. |
| Pipeline control | Less responsibility for operating underlying components; confirm that the available configuration fits the use case. | More control over ingestion, parsing, indexing and storage configuration. |
| Connectors and document-level permissions | Includes capabilities such as connectors and document-level permissions, with exceptions; AWS notes the Web Crawler connector as an exception to document-level permission support. | Some capabilities, including certain third-party connectors and document-level permission features, are available only for Managed Knowledge Bases. |
| Operating burden | Less infrastructure to operate, but the application still has to implement and verify its identity and response policies. | Your team owns more of the pipeline, including the components and controls it chooses to build. |
| Best-fit question | Do the managed ingestion, connector and permission features meet your requirements? | Do custom parsing, indexing, retrieval or vector-store controls justify operating more of the pipeline? |
Neither option is universally better on performance or cost: the available evidence does not establish a numerical winner. Confirm required connectors, permission behavior, regional availability and governance fit against the live service documentation before committing to a design.
Make employee authorization part of retrieval
Employee sign-in and AWS service permissions solve different problems. IAM governs access between AWS principals and services; it does not automatically express which passages a particular signed-in employee may receive. The application must preserve relevant user or policy attributes through retrieval so unauthorized content never reaches the model context.
AWS Prescriptive Guidance describes carrying application user identity into knowledge-base metadata so retrieval can apply metadata filters. A separate AWS Architecture Blog pattern evaluates policy with Amazon Verified Permissions and translates the decision into a metadata filter for Bedrock retrieval. Treat this as one pattern to assess—not a mandatory service or a substitute for validating the permission model against your own identity and document systems. See AWS guidance on Bedrock security integration and the multi-tenant RAG pattern.
Prove the boundary, not just the filter configuration
- Map source permissions and group membership to the attributes used by retrieval; define what happens when a permission is missing, stale or ambiguous.
- Test users from different departments, roles and tenants against documents they should and should not be able to retrieve.
- Verify the actual passages returned to the application and confirm that unauthorized passages cannot enter model context.
- Log enough protected evidence to investigate authorization decisions without unnecessarily exposing sensitive document content.
- Re-test when source permissions, identity mappings, metadata, retrieval settings or policy logic changes.
Protect the full data path
RAG introduces risks at ingestion, retrieval and generation. A malicious instruction embedded in an indexed document can become an indirect prompt-injection attempt; a guardrail at model invocation does not turn that content into trusted input. AWS discusses these risks and layered controls in its guidance on secure access to data and systems for generative AI.
At ingestion
- Accept only approved sources and validate ownership, provenance, file type and source permissions.
- Inspect and filter content before indexing, including for malicious or irrelevant instructions that could influence a model.
- Keep source identity, permission metadata and document lineage linked to indexed content so changes and removals can be managed.
In storage and transit
Set access boundaries and encryption to match the data and the chosen architecture. AWS documents KMS options for knowledge-base resources; TLS protection for communication with third-party connectors or vector stores depends on the provider supporting TLS. Confirm the applicable scope in the Knowledge Base encryption documentation.
At inference and response
Use narrowly configured safeguards to assess inputs and model responses where appropriate. Bedrock Guardrails can be used with Knowledge Bases, but they do not replace retrieval authorization, trusted content handling or answer evaluation. Contextual grounding controls may reduce some risks, not eliminate prompt injection. AWS recommends continued testing and validation in its Guardrails documentation.
In operations
Apply least-privilege IAM, use private network paths where required, audit API activity and monitor both AWS services and application behavior. AWS describes security as a shared responsibility: what the customer must secure depends on the services used, the data, applicable requirements and laws. A service choice alone does not establish privacy, correctness, compliance or immunity to attacks.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Evaluate retrieval separately from answers
A convincing response can conceal a retrieval failure, and a good retrieval result can still be summarized incorrectly. Evaluate these stages separately with a versioned set of representative questions, expected supporting passages and expected answers. Bedrock supports retrieve-only and retrieve-and-generate RAG evaluation jobs, with measures that include context relevance and coverage as well as generated-response evaluation. Start with AWS’s evaluation overview and Knowledge Base retrieval evaluation metrics.
Build a useful evaluation set
- Include routine questions, questions with multiple relevant sources, and questions whose answer is not present in the corpus.
- Add stale or conflicting documents and permission-boundary cases that check whether retrieval excludes unauthorized passages.
- Include adversarial examples, such as untrusted instructions in content, and check whether the system refuses or safely handles them.
- Assess answer correctness and grounding, source relevance and coverage, refusal behavior, and permission correctness—not just a single aggregate measure.
Investigate whether each failure originates in source quality, parsing or chunking, metadata, retrieval settings, authorization, prompt design or generation. Re-run the versioned set after changing any of those components, embeddings, guardrail configuration or model selection. Use human review for consequential workflows; an aggregate score alone is not evidence of readiness.
Bedrock evaluation jobs require access to supported evaluator models; retrieve-and-generate jobs also require the response generator model. AWS documents that both must be available in the same Region. Verify currently supported models and regional availability when implementing, because that constraint can change.
Free tools Windows power users keep installed
One-click scans. No signup required.
Plan for day-two operation
Before launch, assign owners and define observable behavior for the system, not only its cloud resources. Workload-specific capacity, cost and service targets must be established from the organization’s own requirements; no general numerical SLA or cost estimate follows from this architecture.
Quick Recap
- Freshness and lineage: Track ingestion success, failures, document updates and removals so operators can identify what content informed an answer.
- Availability and latency: Set targets for the complete employee-facing path, including identity resolution, retrieval and model invocation; monitor each stage to localize delays.
- Cost and capacity: Attribute retrieval and model usage to the application or workload where possible; account for rate limits and define behavior when capacity is constrained.
- Failure handling: Decide what the user sees when retrieval is empty, source access fails or model invocation is unavailable. A safe response should not invent an answer or silently bypass authorization.
- Audit and incident response: Protect logs, define who can inspect them, and prepare procedures for permission leaks, malicious content, stale data and service disruptions.
Production readiness checklist
- Source owners, approved repositories and content validation rules are defined.
- The managed-versus-customer-managed choice fits connector, permission, customization and operational needs.
- Employee identity and policy attributes reach the retrieval authorization decision.
- Cross-user and cross-department tests verify no unauthorized passage reaches model context.
- Encryption, IAM, network access, logging and incident responsibilities are documented.
- Input and output safeguards are configured for the use case and tested against malicious content and misuse.
- A versioned evaluation set covers retrieval, answer grounding, authorization, refusals and unanswerable questions.
- Freshness monitoring, failure fallbacks, cost attribution and operational ownership are established.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
