GitLab AI Gateway is a standalone service that routes GitLab Duo AI features to model backends; it is not necessarily where the model runs. GitLab operates a gateway for GitLab.com, GitLab Self-Managed, and GitLab Dedicated, while GitLab Self-Managed customers can also deploy their own gateway. The key security question is therefore not just where the gateway lives, but where each feature sends its request next.
What the AI Gateway does
The AI Gateway provides a shared access and routing layer between GitLab and the models used by GitLab Duo. In a managed setup, GitLab operates the gateway and connects it to external model providers. In a self-hosted setup, the customer operates the gateway and configures the model endpoint. In either case, the gateway and the model are separate components and may sit in different infrastructure or trust boundaries. GitLab’s AI Gateway documentation and its AI architecture overview describe the service and its role.
Three request paths
- Managed: GitLab instance → GitLab-hosted AI Gateway → GitLab-managed external model provider → response through the gateway.
- Self-hosted: GitLab instance → customer-operated AI Gateway → configured model endpoint → response through the gateway.
- Hybrid: Routing depends on the model configured for each feature. Features using GitLab-managed models go through GitLab’s hosted gateway; other configured features can use the customer’s gateway and models. GitLab’s self-hosted models documentation explains these deployment patterns.
This separation matters: running the gateway on your own infrastructure does not mean the model runs there too. A self-hosted gateway can connect to cloud services such as AWS Bedrock or Azure OpenAI, leaving the model endpoint outside your network. GitLab’s configuration documentation describes provider and feature configuration.
Where managed requests go—and what regional routing means
For the managed gateway, GitLab documents automatic routing using Cloudflare and Google Cloud Platform load balancers. Latency and availability affect which gateway deployment handles a request; customers cannot choose a region manually, and a request is not guaranteed to go to or remain in one region. The model provider may process it in a different region from the gateway. GitLab’s documentation states: “This service is not a data residency solution.” See GitLab’s regional-routing and residency explanation.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →#1 Best Overall
GitLab lists managed deployments across North America, Europe, and Asia Pacific, but the list can change. Consult the live service manifest linked from GitLab’s gateway documentation rather than treating a static region list as current or as a residency commitment.
Choose a deployment by the boundary you need
The practical differences are who operates each component, where request content can travel, whether internet access is needed, and who maintains the stack.
| Deployment | Gateway and model location | Connectivity and boundary | Who operates it |
|---|---|---|---|
| GitLab-hosted gateway with GitLab-managed models | GitLab operates the gateway, which connects to external model providers. | Requires internet connectivity; requests use GitLab-managed infrastructure and provider services. | GitLab sets up and maintains the managed infrastructure. |
| Fully self-hosted gateway and models | Customer operates both in its own infrastructure. | Can run in an isolated network, subject to the selected supported models and deployment. | Customer hosts, configures, and maintains the stack. |
| Hybrid, configured by feature | Customer operates a gateway and models for some features; other selected features use GitLab-managed models and the hosted gateway. | Features routed to GitLab-managed models require internet connectivity and are not part of a fully isolated deployment. | Customer operates its infrastructure and chooses which features use each path. |
These distinctions are described in GitLab’s gateway documentation and its self-hosted models guide. GitLab records hybrid configuration as generally available beginning in GitLab 18.9 and self-hosted models as generally available beginning in GitLab 17.9; those are release-history details, not a guarantee of current entitlement. Verify the current GitLab tier, license, and supported-model requirements for the version you plan to deploy.
Does self-hosting keep data inside your network?
Only a deployment whose entire request path stays within the intended boundary can meet that goal. A customer-hosted gateway pointed at a cloud model endpoint still sends requests to that external provider. A hybrid configuration also sends features assigned GitLab-managed models to GitLab’s hosted gateway and requires internet access. GitLab documents a fully self-hosted gateway and models as capable of operating in an isolated network, but this depends on the supported model and deployment configuration. For offline installations, GitLab’s instructions require operators to transfer the gateway image, model weights, inference-server image, and other required platform images into internal infrastructure. Licensing and add-on requirements depend on the release. Read GitLab’s offline deployment instructions.
Regional routing, gateway location, model-provider location, and contractual data-residency commitments are separate questions. The managed gateway’s multi-region routing does not guarantee processing in a chosen region; the available documentation does not establish a legal or contractual residency conclusion for a particular customer.
Authentication and network security boundaries
JWT keys and model credentials
In the documented self-hosted authentication flow, the GitLab instance mints the token and the AI Gateway verifies it against the instance. Installation requires separate key pairs for AI Gateway JWTs and Duo Agent Platform JWTs. Each pair includes signing and validation keys; GitLab specifies RSA 2048-bit PEM private keys. The validation key supports rotation so tokens signed with the previous key remain valid until they expire. Treat these keys as sensitive credentials: missing keys prevent token issuance. Administrators can also configure a model API key for authentication to the model provider. GitLab’s installation guide covers the key setup.
Restrict outbound access
GitLab advises restricting outbound traffic from the gateway container and blocking destinations other than the documented requirements: the GitLab instance URL, configured model-provider endpoints, and customers.gitlab.com for license validation unless the deployment uses an offline license. Validate firewall rules outside production first; rules that are too restrictive can break the service. See the gateway installation guidance on network access.
Protect traffic and keep images current
Use TLS for production connectivity to GitLab. GitLab’s Helm chart documentation recommends internal TLS for end-to-end encryption from client to pod; ingress, exposure, and port choices depend on the chart and version in use. Use stable image tags matched to the GitLab version rather than nightly builds, for which backward compatibility is not guaranteed. GitLab also provides a FIPS-validated image option for environments that require FIPS 140-3 validated cryptography. Follow the current installation instructions for patching and image verification. Consult the current installation guide.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesDeployment and operational details
Container requirements and ports
GitLab documents Docker and Kubernetes/Helm installation and describes a combined image with the required code and dependencies. For the documented linux/amd64 container, GitLab lists an approximately 340 MB compressed image, a 512 MB minimum RAM requirement, and access to at least two CPUs for the AI Gateway and Agent Platform services; it says the gateway does not require a GPU. These are published prerequisites, not production capacity or performance guidance. In the documented container setup, AI Gateway HTTP communication uses port 5052, while Duo Agent Platform uses gRPC on port 50052. Confirm ports and exposure against the exact deployment guide and chart version. GitLab’s installation documentation provides the container details.
Plan offline deployment as an image and model supply chain
An offline environment needs more than a gateway container: GitLab’s procedure calls for manually transferring the gateway image, model weights, inference-server image, and other platform images required by the deployment. The specific set of artifacts and licensing conditions should be checked against the release being installed. GitLab’s offline deployment guide describes the process.
Treat examples as examples, not production designs
GitLab’s AWS Bedrock BYOM example places GitLab and the gateway together on one EC2 instance and identifies that architecture as suitable for proof of concept and evaluation. It directs production deployments to reference architectures, so the example should not be treated as production sizing guidance. See the Bedrock BYOM deployment guide.
Quick Recap
A practical boundary check before enabling a feature
- Identify the model configured for the feature and whether it is GitLab-managed or customer-configured; hybrid routing is feature-specific.
- Record separately where the GitLab instance, gateway, and model provider run.
- Map required internet routes and provider endpoints, then restrict gateway egress to the documented destinations.
- Decide whether the requirement is network isolation, a specific processing region, or contractual residency; the managed gateway’s regional routing does not promise the latter two.
- Assign owners for JWT key rotation, model credentials, TLS, image updates, and deployment-specific firewall and ingress changes.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
