A production AI agent is not just a chatbot with a longer prompt. It is an application that puts a model inside a bounded action loop: the model proposes what to do, orchestration and tools carry out approved actions, and the system observes results, handles failures, and returns a useful outcome. To ship one safely, design the application, model access, orchestration, tools, data, execution environment, and operational controls as distinct responsibilities.
What changes when a chatbot becomes an agent?
A chatbot primarily responds to a user. An agent can take actions toward a goal and use what happens next to decide what to do. The essential change is the action loop—not simply a more elaborate prompt or a different model.
As an Amazon Associate I earn from qualifying purchases.
- Reason: interpret the request and decide whether to answer, ask a question, or propose an action.
- Act: request an approved tool or application function, or use an execution environment if the task requires one.
- Observe: receive the action result and update the task state.
- Continue or stop: decide whether another bounded action is needed, whether to seek clarification, or whether to return the result.
Google Cloud describes agents as reasoning, taking action, and observing results, with orchestration managing state, retrieval, and tool use. The model can contribute to decisions, but it does not by itself provide the complete production system.
What belongs in a production agent architecture?
Think of the architecture as a set of cooperating responsibilities rather than a single model endpoint. AWS Prescriptive Guidance presents layered architecture patterns; OpenAI’s Agents API documentation distinguishes an application server, harness, and environment. These are provider-specific views, not a universal deployment blueprint.
User-facing application
The product UI or business application accepts the user’s goal, presents progress and results, and mediates interactions such as approval or clarification. It may also expose existing business functions as tools. Keep user experience and business rules in the application rather than relying on prompt text to enforce them.
Application server and harness
The application server submits work, receives events, and handles application-owned functions. In OpenAI’s documented Agents API model, the harness runs the model-and-tool loop and maintains a session, while the application integrates the agent into the product. This decomposition helps clarify responsibilities; it does not mean every deployment must use a separate server or the same harness design.
Model access
The model interprets the request and proposes responses or actions. Treat access to it as a controlled service: apply the relevant policy and safety checks, and account for usage and cost. The model’s proposed action should not automatically become an authorized external action.
Free tools Windows power users keep installed
One-click scans. No signup required.
Orchestration and state
Orchestration controls task flow, tool selection, state updates, retrieval, and any collaboration among agents. A session may need short-lived state to complete the current task. Long-term memory is a separate data and retention decision, not a requirement for every agent; define what is stored, for how long, and under whose authority.
Tools and knowledge
Tools connect an agent to operations such as looking up a record or submitting a transaction. Knowledge services retrieve relevant enterprise information. Both create security boundaries: apply identity-aware authorization, least privilege, and an audit trail to external actions and data access. AWS guidance specifically calls out secure tool discovery and execution, model access policies, guardrails, and role-based access to knowledge sources.
Execution environment
Some tasks need compute, files, or network access; others can be completed through remote tools without giving the agent its own compute or files. OpenAI’s architecture documentation distinguishes hosted and self-hosted environments. If an environment is self-hosted, the application owns provisioning and lifecycle management. Give an agent only the environment capabilities its task requires.
Cross-cutting operational controls
Authentication, authorization, safety policy, observability, evaluation, resilience, and cost management span the architecture. AWS’s resilience framework treats the foundation model, orchestration, deployment infrastructure, knowledge bases, tools, security and compliance, and evaluation and observability as parts of the system that must work together. Monitoring only the model endpoint leaves important failure paths invisible.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Which workflow shape should you choose?
Start with the simplest workflow that can reliably handle the task. A deterministic workflow may be enough when the steps and branches are known. A single agent can help where a model must choose among bounded tools or handle variation. Multi-agent coordination is an option for a concrete need, not a default sign of production readiness. The sources document patterns rather than a universal performance winner.
| Workflow shape | What it does | Consider it when | Design concern |
|---|---|---|---|
| Deterministic workflow | Application logic directs a known sequence of steps and branches. | The task has stable rules and predictable paths. | Represent exceptions and recovery in application logic rather than expecting a model to improvise. |
| Single agent with bounded tools | A model selects among permitted actions within an orchestrated loop. | The task varies enough that model-led selection is useful, but its actions can be limited and checked. | Control tool permissions, validate inputs and results, and define when to stop or ask the user. |
| Multi-agent coordination | Agents communicate or divide work under an orchestration design. | A specific workflow benefits from distinct roles or coordination. | More handoffs add coordination and failure paths; make responsibilities and recovery explicit. |
Google Cloud’s architecture catalog includes both single-agent and multi-agent patterns. Its documentation describes MCP as an interface for connecting to external data sources and stateless tools, and A2A for communication among agents. Consider such protocols when they solve an integration or interoperability need; their existence does not make them mandatory components.
Rank #4
How do you choose who operates the runtime?
Execution ownership is a design decision with consequences for provisioning, recovery, and shutdown. Compare the responsibilities of the available approaches before choosing one; the names and capabilities of managed offerings vary by provider.
| Approach | Execution ownership | Questions to settle |
|---|---|---|
| Fully self-managed | Your team manages the execution infrastructure and its lifecycle. | Who provisions capacity, isolates tasks, recovers from failure, and shuts resources down? |
| Managed server-based | A provider manages some runtime or server operations; the exact division depends on the service. | Which operational duties remain yours, and what are the service’s limits and recovery behaviors? |
| Serverless or agent-specific runtime | The provider supplies an execution model intended to reduce direct infrastructure management; controls vary. | How are identity, state, networking, observability, and lifecycle handled for this workload? |
OpenAI’s documentation also notes that not every agent needs its own compute or files. Choose an environment based on the workload’s actual needs, not on the assumption that every agent requires a dedicated sandbox.
How should you secure tools, data, and actions?
Treat every tool and knowledge source as a separately authorized capability. A model’s decision to request an action is not proof that the user or agent is entitled to perform it.
- Establish identity: decide which user, service, or delegated identity is used for each operation, and make that identity visible in audit records.
- Apply least privilege: expose only the actions and data needed for the task, with authorization at the level the operation requires.
- Validate at the boundary: check tool arguments and enforce business rules in the tool or application that performs the action.
- Control sensitive actions: require appropriate confirmation or review where the consequences warrant it; do not treat a model-generated rationale as authorization.
- Filter retrieved data: preserve access controls when searching enterprise knowledge, rather than returning information simply because it is relevant.
- Record outcomes: capture the requested tool, authorization result, execution result, and relevant errors so actions can be audited and evaluated.
- Define memory deliberately: set the scope, access, and retention of persistent state instead of assuming that storing more context is always useful.
AWS’s architecture guidance places security, discoverability, and observability across layers. That is a useful design principle: a permission check or audit trail should not depend solely on what the prompt says.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How do you evaluate an agent before and after release?
Assess the path the agent took, not just whether the final response sounds plausible. Google Cloud identifies tool selection, reasoning quality, error recovery, and clarification behavior as evaluation concerns. AWS recommends combining traditional system metrics with agent-specific signals, including traces and tool invocation results.
Build representative trajectories
Create test cases that cover ordinary requests, ambiguous requests, denied actions, tool failures, unexpected tool results, and cases where the agent should stop or ask for clarification. Record expected outcomes and acceptable alternatives. Evaluate intermediate decisions and actions as well as the final answer.
Inspect both agent and infrastructure signals
- Agent behavior: whether it chose an appropriate tool, supplied valid arguments, handled errors, respected access boundaries, and asked for clarification when needed.
- System behavior: latency, availability, resource use, and failures in the model, orchestration, tools, knowledge services, and execution environment.
- Trace completeness: whether you can follow the request through decisions, tool calls, results, retries, and the user-facing outcome.
Re-run evaluations when the system changes
Test representative trajectories before and after changes to prompts, models, tools, retrieval, orchestration, or permissions. A change in one layer can alter the behavior of the whole loop, so a final-answer check alone may miss a new unsafe or ineffective tool path.
What should you verify before adopting a vendor-specific architecture?
Cloud-provider architecture guides are useful explanations of patterns within their own ecosystems, but they do not establish a neutral standard or prove that one stack will perform best for your workload. Compare options using criteria that match the system you need to operate.
- Execution ownership: who provisions, monitors, recovers, and shuts down each runtime component?
- Permissions: how are user identity, tool authorization, data access, and audit records enforced?
- Workflow fit: does the task need fixed application logic, one bounded agent, or actual coordination among agents?
- Evaluation and observability: can you inspect traces, tool outcomes, latency, and infrastructure signals, and test representative trajectories?
- Context and data: how do retrieval filters, memory scope, and retention requirements work?
- Integration and portability: which model and runtime constraints apply, and would an interface such as MCP or A2A address a real integration need?
What is changing in OpenAI’s AgentKit availability?
OpenAI’s AgentKit announcement page was updated June 3, 2026, to state that Agent Builder and Evals are scheduled to become unavailable on the OpenAI platform starting November 30, 2026. The same update points to the Agents SDK for workflows that should continue as code and Workspace Agents in ChatGPT for use cases suited to natural-language prompting. Because the stated end date is approaching, verify current availability and migration guidance before choosing those products; this product lifecycle notice is not an architectural principle.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitches
