Free and open first
Best LLM Gateway Software in 2026
An LLM gateway sits between applications and language-model providers, so routing and administration are useful points to compare. Check provider count and self-hosted deployment, then weigh fallback routing, budget controls, usage analytics, and virtual API keys against your development needs. Bifrost, Routerly, and YoloRouter are among the options shown. Free-plan availability and paid-from pricing add cost context. Consider which combination of provider access, usage visibility, and controls fits the way you plan to connect models in your workflow.
- 34 apps
- Research read 23 Sept 2026
- How the order is worked out
- 1
Bifrost
Free plan Open source APILinuxMac+3Bifrost brings provider access, traffic controls, and observability into one gateway, with a free self-hosted OSS edition.
- 2
LLM Gateway
Free plan Open source APIExtensionLinux+2LLM Gateway combines provider routing, retry handling, and usage analytics behind one API.
- 3
Manifest
Free plan Open source APILinuxSelf-hosted+1Manifest offers both cloud and self-hosted gateway options, with routing, fallback, repair, and spending visibility.
- 4
YoloRouter
Free plan Open source APILinuxMac+3YoloRouter centralizes requests across supported model providers and includes controls for routing, access, and usage.
- 5
Envoy AI Gateway
Free plan APILinuxMac+1Agent Router provides a unified interface and traffic controls for teams managing AI models and tools.
- 6
Helicone
Free plan APISelf-hostedWebHelicone brings routing and observability tools together, with a free tier and self-hosting options. Its maintenance-mode status is worth weighing when considering it for a new deployment.
- 7
LangSmith
Free plan APILinuxMac+3LangSmith combines agent evaluation with prompt tools and observability, including cost and trajectory tracking.
- 8
Cloudflare AI Gateway
Free plan APIWebCloudflare AI Gateway provides a free web gateway with analytics, caching, rate limiting, fallback routing, and budget controls. Its core plan is available on all Cloudflare plans.
- 9
LiteLLM
Free plan APISelf-hostedWebChoose LiteLLM if your platform team needs a shared gateway for many providers, with routing and spend controls.
- 10
Apache APISIX
Free plan APILinuxSelf-hostedAPISIX combines API and AI gateway functions with broad protocol and plugin support. Its self-managed model and separate commercial support are important considerations for teams evaluating it.
- 11
Inference Gateway
Free plan APILinuxMac+3Inference Gateway centralizes provider access, routing, MCP tools, and agent workflows behind one API. It is free, open source, and deployable through Docker, Kubernetes, or self-hosted environments.
- 12
Orq.ai
Free plan APISelf-hostedWebChoose Orq.ai if your team needs a gateway to route and govern model requests while tracking costs and usage.
7.1 Visit site - 13
Polyrouter
Free plan APISelf-hostedWebChoose Polyrouter if you need one endpoint for multiple model providers and want routing, fallback, and budget controls in a free, self-hosted gateway.
- 14
Routerly
Free plan APILinuxMac+3Routerly brings provider routing, cost tracking, and budget controls into a self-hosted gateway. It is free under AGPL-3.0 and adds no markup to API calls.
- 15
SGLang
Free plan APILinuxMac+1SGLang combines model serving, multiple inference optimizations, and broad hardware support in a free framework.
- 16
TensorZero
Free plan APILinuxSelf-hosted+1TensorZero brings gateway, observability, and experimentation capabilities together in a self-hosted, open-source stack.
- 17
TrueFoundry AI Gateway
Free plan APILinuxSelf-hosted+1Choose TrueFoundry AI Gateway if your team needs centralized model access, routing and usage controls across providers.
- 18
OpenRouter
Free plan APIWebOpenRouter brings model access, routing and integrations together behind a single endpoint. The Free plan has a daily request cap, while pay-as-you-go plans apply platform fees to credit purchases.
- 19
Requesty
Free plan APIExtensionWebRequesty combines model access, routing, usage analytics and governance in a managed gateway. Check its logging defaults and pricing structure against your needs before choosing a plan.
- 20
Ramp Router
Free plan APIWebRamp Router combines cross-provider model access with routing, fallbacks, and request-level usage tracking.
6.9 Visit site - 21
Tyk AI Studio Proxy
Free plan APISelf-hostedWebTyk AI Studio Proxy combines routing, guardrails, and usage governance in a self-hostable Community Edition.
- 22
WSO2 AI Gateway
Free plan APILinuxMac+3WSO2 AI Gateway combines provider access, policy controls, analytics and MCP governance in one gateway.
- 23
UniterRouter
Free plan APIWebUniterRouter brings model access, routing, and usage analytics together behind an OpenAI-format API. The free plan offers $5 in credits, while Pro costs $50 a month and Enterprise pricing is custom.
- 24
- 25
vLLM Semantic Router
Free plan APILinuxMac+1vLLM Semantic Router provides policy-driven model selection across deployment locations and backends.
#1Places #1, #2, #3, #4, #5 on this list can be bought by the makers of the software on it. See the price
More in Developer Tools
- LLM Observability Tools26
- LLM Security Tools31
- Load Balancing Software32
- Local Kubernetes Tools23
- Log Management Software111
- Machine Learning Model Monitoring Software20
- Message Broker Software39
- Metrics Monitoring Tools37
- ML Experiment Tracking Software39
- MLOps Platforms29
- Mobile Application Security Testing Software21
- Mobile App Monitoring23