Envoy AI Gateway

Envoy AI Gateway, now called Agent Router, is an open-source control plane for routing AI and agent traffic through Envoy. It suits teams that share models or tools and need centralized credentials, quotas, usage attribution or failover. Its strongest case is self-managed control across many providers; running locally is narrower than production deployment, which depends on Kubernetes.

Overview

Agent Router presents an OpenAI-compatible API for hosted and self-hosted models as well as MCP servers. Its provider integrations span OpenAI, Azure OpenAI, Google Gemini, Vertex AI, AWS Bedrock, Mistral, Cohere, Groq, Together AI, DeepInfra, DeepSeek, Hunyuan, SambaNova, Grok and Anthropic. That breadth makes it a plausible shared entry point for teams working across providers, rather than a model or inference engine of its own.

The project was founded in 2024 and is now an Agentic AI Foundation project. Its site says the code, maintainers and APIs remain the same after the rename. It uses the Apache 2.0 license, with help directed to community Discord and GitHub issues.

Key features

Traffic management

Model virtualization, provider failover, token quotas and usage-based rate limits give teams ways to route requests and govern shared consumption. Usage analytics and budget controls are also supported. These controls suit organizations that need centralized policy; teams seeking only a basic proxy may find the control-plane and deployment requirements more than they need.

MCP and inference routing

The MCP gateway can multiplex servers, route tools, handle OAuth authentication and provide observability. InferencePool support adds intelligent routing and load balancing for inference endpoints. Together, those capabilities make the product relevant to teams managing tools alongside models, not only model API traffic.

Credentials and monitoring

Upstream authentication secures connections to AI services while allowing provider credentials to stay out of application workloads. Credentials can be centrally managed, with short-lived credential flows supported for AWS Bedrock, Azure OpenAI and GCP Vertex AI. Metrics collection and monitoring cover AI workloads, though the project does not replace the need to operate the gateway itself.

Pricing

Open-source project — free. The plan is self-managed, with a local CLI for Linux and macOS and Kubernetes deployment subject to compatible dependencies. The Apache 2.0 project is the fit for teams able to run their own infrastructure and manage its upgrades; there is no paid managed tier described for Agent Router. The project also links to hosted services offered by other providers, which are separate options.

Platforms

The aigw run CLI supports Linux and macOS for local standalone use. Production deployment uses Kubernetes with Envoy Gateway. Agent Router v1.2.x requires Kubernetes v1.32 or later, Envoy Gateway v1.9.2 or later, and Gateway API v1.6.1. Teams on incompatible versions should account for that prerequisite before choosing it for production.

Who it's for

Choose Agent Router if multiple applications or teams share models and tools and need central credentials, usage attribution, quotas or provider failover. It is also suited to self-hosting teams that want one OpenAI-compatible interface across a broad provider set. It is a weaker fit for users who need a managed gateway, broad local operating-system support, or production deployment without Kubernetes.

Pros and cons

Pros

• Broad provider coverage: One compatible API spans hosted and self-hosted models and MCP servers across numerous named providers.
• Useful governance controls: Quotas, usage-based limits, budget controls and failover address shared traffic and provider resilience.
• Credential isolation: Central management and supported short-lived flows can keep upstream credentials out of application workloads.
• Open-source self-management: Apache 2.0 licensing and a free plan suit teams that want to operate the gateway themselves.

Cons

• Production has infrastructure prerequisites: Kubernetes and compatible Envoy Gateway and Gateway API versions are required.
• Local support is limited to Linux and macOS: Teams seeking a local Windows CLI will need another option.
• Self-managed operation: The free project leaves deployment and ongoing operation to the team, with community Discord and GitHub issues as support routes.

Alternatives

LLM Gateway Software is the broader category directory for comparing gateway options.

• Bifrost is worth comparing for teams wanting an OSS gateway with self-hosting through npx, Docker, Kubernetes or a Go binary; its free plan is billed free forever.
• Routerly offers a free self-hosted plan with no markup on API calls, a useful alternative for teams prioritizing that pricing model.
• YoloRouter offers a self-hosted single binary with SQLite out of the box and optional PostgreSQL, for readers preferring that deployment format.
• Kong Gateway may suit teams seeking a time-limited trial: its free trial lasts 30 days, requires no credit card and has no Gateway limits.
• SGLang is an open-source inference framework installable with pip or Docker, rather than a direct gateway choice.
• TensorZero is an alternative for teams looking for a 100% self-hosted, open-source LLMOps platform.
• Eden AI charges per API call at provider prices plus a 5.5% platform fee when purchasing credits, making it a different option for readers evaluating that usage model.
• Helicone describes itself as open source and offers a free Hobby plan with 10,000 free requests.

Verdict

Agent Router is a strong choice for infrastructure-capable teams that need a self-managed, OpenAI-compatible control point for models and MCP tools, especially when centralized credentials, quotas and failover matter. Its breadth and governance controls are the reasons to choose it; look elsewhere if Kubernetes production prerequisites or Linux/macOS-only local use do not fit your environment.