DriversRecommendedOutdated drivers can make a good PC feel brokenScan driver issues before chasing fixes manually.Scan NowHalloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsSlow PC?RecommendedPC slow today? Run a repair scan before it gets worseResolve common Windows issues and optimize system performance.Scan Now×
FREEDOM251

Freedom251 › Blog › Amazon EKS

Build a Production-Ready Kubernetes Infrastructure on AWS with Terraform: Beyond the 10-Minute EKS Quick Start

By Freedom251 Team · · 9 min read

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Terraform can create an Amazon EKS cluster in a single run, and the cluster step itself can be quick. Production readiness takes longer, because it depends on decisions that sit around the cluster: subnets sized for pod IP addresses, Terraform state that a team can share safely, a compute model you can operate and upgrade, explicit security ownership, and monitoring that reaches a named person. The ten-minute figure in the title is a hook rather than a measured result. AWS’s Terraform AI/ML sample describes its self-managed Karpenter route as taking about 15 minutes, and that estimate applies to that sample and its workload only.

What “EKS in 10 Minutes” covers, and what it leaves open

AWS’s application-ready EKS guidance is a larger build than a quick start. It runs several Terraform stages and deploys networking, scalability, observability and security components as one integrated architecture. A cluster that comes up quickly is a working cluster. A production-ready one has also been designed, secured, observed, upgraded and validated for a specific workload.

As an Amazon Associate I earn from qualifying purchases.

Quick starts often leave five areas open. Each one is covered below.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • Network capacity: Availability Zones, subnet size and pod IP headroom.
  • State and access: where Terraform state lives, and who can read it or run changes against it.
  • Compute lifecycle: how nodes are added, replaced and upgraded.
  • Security boundaries: IAM, pod-to-pod traffic and access to AWS services.
  • Observability and ownership: what is measured, what raises an alert, and who responds.

Plan network capacity before you choose node sizes

Subnet space and pod density set how many pods a cluster can run. Settle both before you choose instance types or autoscaler targets, because these limits are easy to reach once workloads are running and harder to change afterwards.

Subnets and Availability Zones

AWS’s networking guidance recommends at least two Availability Zones for cluster subnets, and checking the free IP addresses in each subnet before you deploy. To list subnet capacity in the target account and region:

aws ec2 describe-subnets --query 'Subnets[].[SubnetId,AvailabilityZone,AvailableIpAddressCount]' --output table

AWS’s reference architecture uses three Availability Zones, with VPC endpoints for services such as ECR, EKS, EC2 and EBS. Those endpoints belong to that reference design and are not a universal requirement.

How the default VPC CNI limits pods per node

In the default secondary-IP mode of the Amazon VPC CNI, each pod receives an IP address from the elastic network interfaces (ENIs) attached to its node. The instance type determines how many ENIs and addresses are available, which sets the default ceiling:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Maximum pods per node = (ENIs × (IPv4 addresses per ENI − 1)) + 2

The “+2” accounts for host-networked pods such as aws-node and kube-proxy. An m5.large supports 3 ENIs with 10 IPv4 addresses each, so the formula gives (3 × 9) + 2 = 29 pods per node. That value is calculated from the formula, not measured on a running cluster, and it changes with instance type and CNI settings.

Prefix mode, IPv6 and custom networking

Approach What it changes Trade-off to check
Secondary-IP mode (default) Pods take individual secondary IPs from node ENIs, so the instance’s ENI and address limits cap pod density Simplest setup; small instance types hit this limit first
Prefix mode AWS notes it can help with pod-density constraints; its guidance does not give a per-instance gain Confirm support for your CNI version and instance types before relying on it
IPv6 AWS points to IPv6 when IPv4 address space is exhausted Your organization must be ready to run IPv6 across its network
Custom networking Places pods in subnets separate from the node subnets AWS identifies added operational overhead

Structure Terraform so state and modules hold up

Local state for experiments, remote state for teams

Local state is a reasonable default for a first run on one machine. HashiCorp recommends remote state for team collaboration and lists Amazon S3 as a supported remote store. Backends that support locking prevent two runs from changing the same state at once. State can contain sensitive values, so never commit it to version control or paste full state files into tickets or chat.

Concern Local state Remote state in S3
Team collaboration Tied to one machine Shared by everyone who has access to the backend
Concurrent runs No protection Locking, where the backend supports it
Access control File permissions on the machine Bucket and IAM policies that you configure
Sensitive data Copies left on laptops or in repositories are easy to create Can be encrypted and restricted, but must still stay out of version control
terraform {
  backend "s3" {
    bucket       = "example-eks-tfstate"
    key          = "prod/eks/terraform.tfstate"
    region       = "us-east-1"
    encrypt      = true
    use_lockfile = true
  }
}

The use_lockfile setting is available in Terraform 1.10 and later and stores a lock object in the bucket. Older configurations lock with a DynamoDB table instead. Check which Terraform version your project requires before you choose. Create the state bucket in a separate bootstrap configuration, because the cluster configuration cannot store its own state before the bucket exists, and turn on bucket versioning so earlier state versions can be recovered.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Modules: group by responsibility

Put networking, compute, cluster add-ons and application components into their own modules, with documented inputs and outputs. HashiCorp recommends modules for reuse but advises moderation. A module that wraps a single resource without adding an architectural abstraction, or modules nested several levels deep, make configurations harder to read and change.

Pin Terraform, provider and module versions

Set required_version for Terraform, declare required_providers with version constraints, pin module sources to specific versions, and commit the .terraform.lock.hcl lock file. Verify exact version numbers against your project’s requirements at implementation time, because EKS, provider and module releases each move on their own schedule.

Build order for the first environment

  1. Create the state bucket in a bootstrap configuration with encryption and versioning enabled. Add the backend block to the cluster configuration. If you are moving from local state, run terraform init -migrate-state.
  2. Run terraform init with the pinned Terraform and provider versions.
  3. Run terraform plan -out=tfplan and read the plan before applying. Look closely at changes to the VPC, subnets and cluster IAM roles.
  4. Run terraform apply tfplan. Applying the saved plan ensures that the changes you reviewed are the changes that run.
  5. Update your kubeconfig with aws eks update-kubeconfig, using the region and cluster name from your Terraform outputs.
  6. Verify with kubectl get nodes and kubectl get pods --all-namespaces. Nodes should report Ready and the system add-ons should be running before you deploy any workload.

Choose the compute model deliberately

EKS Auto Mode

In AWS’s Terraform AI/ML sample, EKS Auto Mode manages several platform capabilities for you. Choose it when you want AWS to own more of the platform components and you are comfortable with less direct control over them.

Self-managed Karpenter

The self-managed Karpenter route has Terraform install and configure networking add-ons, Karpenter and monitoring. You get more direct control over those components, and you also own their upgrades and failure modes.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Reference design: a managed node group for critical add-ons

AWS’s application-ready guidance runs CoreDNS, Karpenter and the AWS Load Balancer Controller on a managed node group. This keeps Karpenter itself off the capacity it provisions. Karpenter then manages capacity for other add-ons and workloads. This is one reference design, not the only valid production topology.

Factor EKS Auto Mode Self-managed Karpenter
Platform components AWS manages several platform capabilities You install and configure networking add-ons, Karpenter and monitoring
Setup time in AWS’s sample Not stated About 15 minutes, for that sample and workload
Switching paths midstream Requires destroying and recreating the cluster, according to AWS’s sample guide

Security: divide ownership before you write policies

AWS’s Best Practices for Security – Amazon EKS documentation states the division directly: “Generally speaking, AWS is responsible for security ‘of’ the cloud whereas you, the customer, are responsible for security ‘in’ the cloud.” AWS manages the managed control plane, including the Kubernetes control-plane nodes and the etcd database. The rest of the stack belongs to you.

Area AWS manages You own
Control plane Kubernetes control-plane nodes and the etcd database Cluster configuration that you declare in Terraform and Kubernetes
Managed node groups Not stated as AWS-performed upgrades Moving node groups to current AMIs, and capacity planning
Identity Not covered by the control plane IAM roles, cluster administration and access levels
Pods and runtime Not covered by the control plane Pod and runtime security, and workload controls
Network security Not covered by the control plane Network policies, security groups and subnet design

Pod traffic: network policies, security groups for pods and service meshes

AWS recommends least privilege for network traffic. Start from a default-deny posture, then allow only the flows each workload needs. The three options below differ in what they control and what they cost to run.

Option Layer What it controls Trade-off
Kubernetes network policies Layers 3 and 4 Pod traffic inside the cluster May be sufficient for simpler isolation needs
Security groups for pods Not stated Pod access to AWS services, using AWS security-group rules Rules live alongside the AWS resources they protect
Service mesh Layer 7 Service-to-service traffic, with traffic management, detailed service telemetry and mTLS Adds resources and operational work

Read the sample’s defaults before you apply

AWS’s AI/ML sample warns that its Grafana ingress defaults can expose the dashboard publicly over HTTP with default credentials unless you restrict the source CIDR. Its guide recommends restricting access and notes that the resources are billable. Before applying any example, check ingress rules, credentials and CIDR ranges, and treat the sample as a starting point rather than a hardened configuration.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Observability: monitor what the business depends on

AWS recommends starting with the infrastructure, application and security metrics most relevant to business reliability, then widening coverage as operational needs become clear. Validate monitoring by confirming that each alert fires under the condition it describes. Then apply these rules:

  • Set alert thresholds from your SLOs and from historical behavior, not from default values.
  • Give each alert a severity tier, a named owner and an escalation path.
  • Set retention and archival rules for logs and telemetry before launch, because they drive both access and storage costs.

AWS identifies several EKS monitoring tools. None is the right choice by default. Choose by what you must measure, what your team already runs, and who will operate it.

Tool Role in EKS monitoring Questions to answer before adopting
Prometheus Collects and queries Kubernetes and application metrics Does the team already run it, and who maintains its storage and retention?
CloudWatch AWS metrics and logs for EKS and related AWS resources What will ingestion and retention cost at your telemetry volume?
CloudTrail Records AWS API activity Which audit events must you keep, and for how long?
ADOT Operator Manages AWS Distro for OpenTelemetry collection Should telemetry flow to existing backends, and who runs the collectors?

Operate the cluster after the first apply

Upgrades and node ownership

Managed node groups need you to move nodes to current AMIs, and they do not scale the cluster automatically the way Fargate does. You need an upgrade routine for node AMIs and a scaling mechanism; the reference design uses Karpenter for that role. Check the Kubernetes versions EKS currently supports before each upgrade, because support windows change. Rehearse the upgrade in a non-production cluster built from the same Terraform code.

Cleanup of test environments

Run terraform destroy against the same state you created, not a fresh local copy, so Terraform removes the resources it tracks. Then check the AWS console for load balancers and volumes created by Kubernetes controllers, which Terraform does not track and which can block deletion of subnets and the VPC. Resources from AWS’s samples are billable, so confirm they are gone.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Symptoms to check first

  • The plan proposes replacing the cluster or node resources. Check whether the compute path changed, and compare the plan with the current state before you apply.
  • Runs wait on a state lock. Confirm that no run is active before clearing the lock with terraform force-unlock. Forcing an unlock during a live run can corrupt state.
  • Pods stay Pending while nodes have spare CPU and memory. Check pod density per node and free subnet addresses. The IP limit is the likely constraint.
  • Nodes do not join the cluster. Check that the node subnets have free addresses and that the node IAM role matches your configuration.

The Bottom Line

Use the quick start to learn the Terraform layout and confirm that the cluster builds in your account. Do not treat the first successful apply as production readiness. Size subnets for pods, move state to a locked remote backend, choose a compute model you are prepared to upgrade, write down who owns each layer, and make sure every alert has a person behind it.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Amazon EKSAWSDevOpsinfrastructure as codeKubernetesTerraform

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Recommended PC Tool
Recommended PC Tool
Outdated Drivers Are Slowing You DownFree scan - exact matches
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.