Recommended Free Tools
Neither a self-hosted nor a cloud-hosted AI gateway is automatically more secure. Self-hosting gives your organization more direct control over the gateway’s infrastructure and data stores, but makes your team responsible for operating and securing them. A managed gateway can simplify that work and centralize routing, but adds the gateway provider to the request and credential trust boundary. The right choice depends on the full flow of prompts, credentials, logs, policies, and model-provider traffic—not on the hosting label alone.
First, separate gateway hosting from model hosting
An AI gateway routes requests between applications and model providers and may add centralized controls such as authentication, logging, caching, or rate limiting. Hosting that gateway yourself does not necessarily mean the model runs on your infrastructure. If your gateway forwards a prompt to an external provider, that provider remains part of the data path.
For example, LiteLLM documents deployment in infrastructure an organization operates, while Cloudflare describes AI Gateway as a route to models hosted by Cloudflare or third parties such as OpenAI, Anthropic, and Google. In either pattern, assess gateway data location separately from where inference happens. LiteLLM production deployment; Cloudflare AI Gateway REST API.
What self-hosting requires
Self-hosting means choosing and operating the gateway environment. LiteLLM’s production deployment guide documents Kubernetes deployment with Helm on EKS, GKE, or AKS, as well as official Terraform modules for AWS and Google Cloud. For Azure, the guide identifies AKS with Helm as the supported path. Its architecture can be a monolithic service or separate gateway, backend, and UI components. LiteLLM production deployment.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errors#1 Best Overall
The production reference architecture uses PostgreSQL for keys, teams, users, spend logs, and configuration; Redis for rate limiting, router state, and cross-instance caching; and managed secrets for master and provider keys. LiteLLM says PostgreSQL is required for proxy authentication and tracking features, and Redis is required when running more than one instance. These components make the organization responsible for deployment, configuration, patching, availability, secret handling, and monitoring.
This gives an organization more direct control over where the gateway runs and how its supporting systems are configured. It does not by itself establish that prompts stay within a private network: a remote model call can still send prompt data to an upstream provider.
What a cloud-hosted gateway changes
Cloudflare’s AI Gateway REST API provides a managed route to models hosted by Cloudflare or third parties, with documented logging, caching, and rate limiting. It supports an envelope endpoint and OpenAI-compatible chat-completions and Responses API endpoints; Responses support depends on the model. Cloudflare handles the gateway service, while the customer still manages account access, application integration, and policy configuration. Cloudflare AI Gateway REST API.
Using the managed service puts its endpoint in the request path. Before sending sensitive traffic, review the current data-handling, logging, retention, and plan terms for the specific configuration. The available documentation describes gateway features, but does not establish a universal retention outcome for every plan or setup.
Security and control comparison
| Decision area | Self-hosted example: LiteLLM | Cloud-hosted example: Cloudflare AI Gateway | What to verify |
|---|---|---|---|
| Gateway infrastructure | Deploy and scale gateway services and supporting database or cache infrastructure in selected cloud accounts or Kubernetes. LiteLLM deployment guide. | Use the vendor’s API endpoint and account-managed service. Cloudflare REST API documentation. | Who hardens, patches, monitors, scales, and responds to incidents in the gateway layer? |
| Prompt and response path | The gateway can run in organization-selected infrastructure, but remote model calls may transmit prompts to an upstream provider. LiteLLM deployment guide. | The managed endpoint documents routing, logging, and caching features. Cloudflare REST API documentation. | Which systems can see request content, and which retain it under the selected configuration? |
| Provider-key custody | The operator must protect configured master and provider keys; LiteLLM’s AWS example uses a secrets manager. LiteLLM deployment guide. | Cloudflare documents Bring Your Own Key (BYOK), so administrators can store provider keys in its dashboard rather than send the provider key on every request. Documented controls include rotation, revocation, multiple keys, and aliases. Cloudflare BYOK documentation. | Who stores each credential, which workloads and people can use it, and how quickly can it be revoked? |
| Authentication and scope | The operator chooses and configures the gateway’s authentication and deployment boundary. LiteLLM documents virtual keys and per-key, team, and user budgets. LiteLLM getting started documentation. | When Authenticated Gateway is enabled, a Cloudflare API token is required. Cloudflare says AI Gateway Read, Run, and Edit permissions are account-scoped, not restrictable to one gateway; it recommends separate accounts or a Worker-side binding for isolation. Cloudflare Authenticated Gateway documentation. | Are tokens scoped to the required account, gateway, action, and workload? Is tenant isolation adequate? |
| Policy and inspection | LiteLLM’s overview documents centralized logging, guardrails, and caching; available controls depend on setup and configuration. LiteLLM getting started documentation. | Cloudflare’s wrapper tutorial documents optional prompt and response guardrails, Access policies, DLP profiles, isolated browser sessions, visibility into prompts, responses, and usage, and log export. Cloudflare AI Gateway and Zero Trust tutorial. | Where are controls enforced: before data leaves the user’s environment, at the gateway, or at the model provider? |
| Operational burden | The organization runs the gateway and its dependencies, including multi-replica and database/cache considerations. LiteLLM deployment guide. | The vendor operates the gateway service; customers still manage account permissions, tokens, application integration, and policy configuration. Cloudflare REST API documentation; Authenticated Gateway documentation. | Does your team have the capacity and operational controls to secure the responsibilities it retains? |
These are documented product behaviors, not an independent security audit or a universal scorecard. LiteLLM and Cloudflare AI Gateway are examples of deployment patterns; their documented properties should not be assumed to apply to every self-hosted or managed gateway.
How to choose for your organization
- Map the data path. Trace prompts and responses from the application through the gateway to the model provider. Identify which parties can inspect or store content at each point.
- Map every credential. Record where gateway tokens and provider keys are stored, which accounts or workloads can use them, how they are rotated, and how revocation works.
- Check authorization and isolation. Verify the actual scope of permissions and whether the design isolates teams, tenants, and gateways sufficiently. With Cloudflare’s documented account-scoped AI Gateway permissions, consider its recommended separate-account or Worker-side binding approaches where gateway-level isolation is needed. Cloudflare Authenticated Gateway documentation.
- Decide who will operate the gateway. Self-hosting requires ownership of the gateway and supporting services; a managed service reduces that infrastructure work but does not remove customer responsibilities for access, integration, and policy.
- Review data and contract terms for the exact configuration. Confirm logging and retention behavior, data processing terms, and model-provider handling for the service, plan, and providers you intend to use. Do not infer these terms from the hosting model alone.
When each model is a better fit
Consider self-hosting when infrastructure control is a priority
A self-hosted gateway may fit when the organization wants to choose the gateway’s deployment environment and data stores and has the people and processes to operate them securely. That choice provides control over the gateway layer, not automatic control over an external model provider’s processing.
Consider a cloud-hosted gateway when reducing gateway operations matters
A managed gateway may fit when a centralized API and vendor-operated gateway are useful, provided the organization is comfortable adding the service to its request and credential trust boundary. Confirm that its permission model, logging and retention terms, and isolation options meet the use case.
Use the architecture, not the label, as the decision
Compare concrete configurations: application-to-gateway traffic, gateway-to-model traffic, credential custody, logs, policy enforcement, tenant isolation, and operational ownership. Neither “self-hosted” nor “cloud-hosted” alone proves that a deployment is private, compliant, or more secure.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
