agentgateway

Overview

agentgateway is an open-source HTTP and gRPC gateway for service, model, and agent traffic. It suits platform teams that want to govern AI workloads on infrastructure they operate. Its broad controls make it a strong fit for self-managed deployments, but not for buyers seeking a hosted gateway.

Key features

An OpenAI-compatible API routes requests to hosted providers including OpenAI, Anthropic, Bedrock, Gemini, and Vertex, as well as open-source Llama runs. For teams running inference themselves, it supports vLLM, TGI, and Triton, with routing informed by latency, cost, and model. That range is useful when workloads span providers and hosting models; teams using a single simple API may not need this gateway’s operational scope.

For agent workflows, MCP server discovery, scoped tool calls, and audit trails sit alongside A2A routing between runtimes such as LangChain, CrewAI, and ADK. MCP guardrails inspect calls and responses for prompt injection and unsafe payloads, while prompt guards support checks and redaction. These are relevant controls for teams exposing tools to agents, though they still require operators to set and maintain suitable policies.

Security features include JWT and OIDC, API keys, authorization policies, TLS and mTLS with certificate rotation, and rate limiting. Observability includes OpenTelemetry, Prometheus token-usage histograms, Jaeger traces, and request logs with latency, token counts, and realized USD cost. Scoped virtual keys and per-key or per-team token and dollar caps help constrain spend; a dashboard groups usage by model, provider, and user. Together, these features give platform teams visibility and controls across traffic, but they come with the responsibility of running and configuring the gateway.

Model integrations include OpenAI, Anthropic, Gemini, Bedrock, Azure OpenAI, Mistral, DeepSeek, Ollama, Vertex AI, xAI, and Snowflake. Deployment can be a standalone binary, Docker container, or Kubernetes workload. The project was created by Solo.io and is an AAIF project; enterprise commercial support is offered through partners, including Solo.io’s 24×7 coverage, upstream collaboration, and best-practice guidance.

Pricing

The open-source gateway costs nothing. The free plan includes the gateway and supports binary, Docker, or Kubernetes deployment. It is a practical starting point for teams able to operate self-hosted software; there is no paid self-hosted tier described for buyers who want a different level of included service. Partner-provided commercial support is a separate option, with custom pricing.

Platforms

agentgateway is available for API and Linux environments and is self-hosted. Its binary, Docker, and Kubernetes deployment choices suit infrastructure teams that want deployment flexibility, but do not make it a managed service.

Who it's for

Tool-builders, platform engineers, and AI enthusiasts are the natural audience, particularly teams handling both model requests and agent-tool traffic. Teams that need routing, security, observability, and spend controls under their own operational control can make use of its breadth. It is a less suitable choice for organizations that want a provider to host and run the gateway.

Pros and cons

• Pro: A free, open-source gateway spans hosted LLM providers, self-hosted inference, MCP, and A2A traffic, making it relevant to mixed AI infrastructure.
• Pro: Security controls, tool-call guardrails, traces, request logs, and spend caps address operational and governance needs in one gateway.
• Pro: Binary, Docker, and Kubernetes options give teams several ways to run it.
• Con: Self-hosting means the team must operate the gateway and its policies rather than relying on a managed service.
• Con: The feature set is oriented toward platform operations; teams seeking only a straightforward hosted API may find the scope excessive.

Alternatives

For a wider set of Kubernetes ingress options, compare the Kubernetes Ingress Controllers directory.

• ngrok is worth considering when a freemium option with a free trial and online endpoints is a better fit; its free plan includes up to 3 online endpoints and up to 1GB data transfer.
• Kusk Gateway is another free, fully open-source option under the MIT License.
• BFE Ingress Controller offers a free Apache 2.0-licensed open-source option.
• Traefik Proxy is a free open-source choice combining ingress control, reverse proxying, load balancing, and service auto-discovery.
• HAProxy Ingress is a free option for Kubernetes clusters; version 0.16 requires Kubernetes 1.21 or newer.
• Higress offers a self-hosted community edition with community support and documentation, while users handle stability and performance.
• NGINX Gateway Fabric is a free Kubernetes deployment with Gateway API support that varies by resource.
• Airlock Microgateway has a community edition for small environments or local development, with enforced throughput and a six-month license validity.

Verdict

Choose agentgateway if your platform team needs one self-hosted layer for model and agent traffic, especially when spend controls and security matter alongside routing. Its breadth and free open-source plan are the case for it; the need to run and govern the gateway yourself is the clearest reason to choose another approach.