There is no defensible universal winner among LLM gateways: the right fit depends on how much infrastructure your team wants to operate, how it handles provider failures, and what visibility and governance you need. Five options discussed in 2026 comparison coverage are LiteLLM, Portkey (now branded on its official page as PRISMA AIRS AI Gateway), OpenRouter, Kong AI Gateway, and Cloudflare AI Gateway. The available comparisons do not establish a common hands-on test of these five, so this is a production-focused selection guide—not a claim that they were independently benchmarked.
What an LLM gateway does in production
An LLM gateway sits between an application and one or more model providers. Depending on the product and plan, it can give an application a unified way to reach models and add request routing, retries or fallbacks, caching, usage and cost telemetry, rate limits, budgets, or governance controls. Those capabilities are not identical across tools, and a feature listed for a product may depend on its deployment mode or plan.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
Gateways at Scale | $0.99 | Buy on Amazon |
| 2 |
|
AI Gateway: The Intelligent Control Plane for AI Systems | $8.00 | Buy on Amazon |
| 3 |
|
AI Routers, Explained: How Smart Systems Choose Models, Control Costs, Reduce Risk, and Keep... | $9.99 | Buy on Amazon |
As an Amazon Associate I earn from qualifying purchases.
The gateway also becomes part of the request path. Its availability, configuration, data handling, and failure behavior matter alongside provider choice. A gateway can simplify integration, but it does not make all models or provider-specific features interchangeable.
Five production options to assess
The table summarizes how 2026 comparison coverage characterizes these tools. It is not a feature-by-feature certification or a substitute for checking current documentation, plan terms, and deployment options. Arize AI says its pricing information was verified on August 31, 2026; that date is not a guarantee that prices or limits remain current.
#1 Best Overall
| Gateway | Deployment and positioning reported | Production considerations |
|---|---|---|
| LiteLLM | Arize AI characterizes it as an open-source, self-hosted gateway with broad provider support. | Arize lists virtual keys, budgets, rate limits, load balancing, retries and fallbacks, caching, and telemetry. Self-hosting means your team owns deployment, capacity, upgrades, monitoring, and availability. |
| Portkey / PRISMA AIRS AI Gateway | Arize characterizes it as managed or hybrid. Portkey’s official product page uses the PRISMA AIRS AI Gateway branding and describes a broader production platform. | Comparison coverage lists routing, retries, fallbacks, caching, logs, traces, and guardrails. Confirm which functions and deployment choices are available for the specific plan and configuration you would use. |
| OpenRouter | Arize characterizes it as a managed service with a large model catalog. | Arize describes routing and fallback options, analytics, and policy controls. Account for model inference charges and any applicable credit-purchase or bring-your-own-key fees; verify current official terms rather than relying on older comparisons. |
| Kong AI Gateway | Comparison sources describe managed and self-managed options and position it as a fit to assess for organizations already operating Kong. | Some advanced capabilities may depend on paid enterprise offerings. Verify the exact license scope and plan requirements before treating a feature as available. |
| Cloudflare AI Gateway | Comparison sources describe it as a managed option associated with Cloudflare’s network. | Vercel’s comparison distinguishes automatic retries for transient upstream errors from cross-provider routing: routing across providers requires separate Dynamic Routing configuration. Check the configured triggers and what failures the application actually sees. |
These profiles reflect descriptions in Arize AI’s 2026 gateway comparison and Vercel’s 2026 comparison, rather than independent measurements. They do not establish that any one option has the lowest latency, lowest total cost, or best reliability for your workload.
Choose the deployment model before comparing feature checklists
Self-hosted
A self-hosted gateway can give a team more control over where the service runs and how it is operated. That control comes with responsibility: provision capacity, deploy updates, monitor health, manage availability, and test recovery. LiteLLM is characterized by Arize as self-hosted, so evaluate it with those operational obligations included—not just its feature list.
Managed or hybrid
A managed gateway can shift much of the gateway infrastructure burden to a vendor; hybrid options may split responsibilities. That can be attractive when the team wants to focus on application behavior rather than operating another service. It does not remove the need to assess data handling, access controls, service dependencies, configuration, and the consequences of a gateway outage. Confirm what is managed and what remains yours for the exact deployment and plan.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsExisting platform ecosystem
If your team already operates a platform that offers a gateway, its integrations and existing operational expertise may reduce adoption friction. Kong is one option to assess in an organization already using Kong; Cloudflare AI Gateway may be relevant to teams already working within Cloudflare’s environment. Ecosystem fit is a decision factor, not evidence that either is superior for a particular workload.
Test failure handling, not just the word “fallback”
Retry, fallback, and routing are related but distinct behaviors. A retry repeats or reattempts a request after a qualifying error. A fallback can send a request to another configured model or provider. Routing selects where a request goes according to a policy, which may be static or conditional. Do not infer cross-provider failover from a product’s mention of retries.
For each candidate, document the failure conditions that matter to your service and verify the resulting request path. Cloudflare is a concrete example: Vercel’s comparison says automatic retries address transient upstream errors, while routing across providers requires separate Dynamic Routing configuration. That distinction should not be generalized to other products without checking their behavior.
- Which errors trigger a retry, a fallback, or neither?
- Does a retry go to the same provider, and what configuration is needed to use another provider?
- Are timeouts, rate limits, provider outages, and malformed responses handled differently?
- Can retries multiply requests, delay a response, or create duplicate side effects in your application?
- What does the caller receive if all configured routes fail, and can operators see which path was taken?
Run these checks with the models, provider accounts, timeouts, and policies your application will actually use. A feature label is not a substitute for verifying triggers, ordering, and application-visible behavior.
Compare routing control and model compatibility against your workload
Start with the routes your application needs, rather than the largest advertised model catalog. Decide whether you need a fixed provider selection, weighted distribution, load balancing, cost- or latency-based policies, or semantic routing. Then confirm that the gateway supports the policy you intend to use and that the behavior is available in your deployment and plan.
Test the API features your application depends on through the gateway. A broad catalog does not prove that every model feature, provider behavior, tool call, or policy behaves consistently through the abstraction. Keep a direct-provider path or a deliberate migration plan if your application relies on provider-specific behavior that the gateway does not preserve.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Measure operational value and total cost
Compare the whole cost of serving requests, not a headline gateway price. Separate gateway subscription or usage fees from model inference charges; include any credit-purchase or BYOK fees, the effect of caching where applicable, and the engineering and infrastructure effort required to operate the service. Prices and plan limits change, so confirm current terms directly with the vendor before committing.
Cost attribution is useful only if it matches how your organization makes decisions. Check whether logs and usage can be separated by application, team, environment, or key; whether budgets, rate limits, and alerts are available; and whether data can be exported or connected to existing operational systems. Review retention, access controls, storage location, auditability, and any guardrails or PII/DLP controls against your own requirements.
Free tools Windows power users keep installed
One-click scans. No signup required.
Latency figures need a shared method to be meaningful. Vercel cautions that tests using mock providers measure gateway forwarding overhead, while a real provider’s response time can dominate the end-to-end result. Measure with a realistic upstream and workload, and distinguish gateway overhead from total response time. Vercel also reports figures for its own gateway, including a first month with roughly 16,000 hours of runtime—1,200 hours of real CPU work and 14,800 hours waiting on provider responses—and, through April 2026, a fallback path that rescued 5.1% of tokens and 4.9% of market cost alongside a 3.5% request figure. These are Vercel-reported figures for its own system, not comparable measurements across the five options here.
A practical evaluation sequence
- Write down requirements. Specify the providers and model features you need, deployment constraints, failure scenarios, governance requirements, traffic shape, and the operational work your team can own.
- Shortlist by deployment fit. Eliminate candidates that cannot meet your managed, self-hosted, hybrid, or organizational requirements. Include the people and infrastructure needed to operate a self-hosted option.
- Verify the exact plan and configuration. For every must-have—such as routing, fallbacks, logs, budgets, guardrails, or export—confirm availability, limits, and prerequisites in current vendor documentation or terms.
- Exercise realistic requests and failures. Send representative requests through the gateway, then test the provider errors and timeouts your service must withstand. Record routing decisions, retries, caller-visible results, and operator visibility.
- Measure end-to-end behavior. Use a realistic upstream and representative request mix. Separate gateway overhead from provider response time, and compare candidates using the same method rather than mixing vendor-reported figures.
- Review the economics and data path. Estimate gateway charges, model inference, fees, cache effects if relevant, and engineering or infrastructure effort. Confirm logging, retention, access, and governance fit before routing sensitive production traffic.
- Roll out with a recovery plan. Set monitoring and limits, decide how traffic can be redirected if the gateway or configuration fails, and validate the rollback path before broadening deployment.
Which gateway should you choose?
Use the deployment model and operational trade-off as the first filter. A self-hosted option may suit a team that prioritizes control and can accept the operating work. A managed service may suit a team seeking less gateway infrastructure burden. A gateway already aligned with the organization’s platform can be convenient. Then let verified failure behavior, routing needs, compatibility, data governance, and total cost determine the shortlist. The available comparison sources do not provide a common-method benchmark that supports naming a universal performance winner.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
