Provisioning Kubernetes Clusters On AWS Using Terraform And EKS

Introduction

Efficient management and orchestration of Kubernetes clusters have become essential to ensuring the availability of modern applications. However, setting up Kubernetes clusters from scratch is laborious. Amazon Web Services (AWS) offers a robust solution for orchestrating Kubernetes clusters with its service, Amazon Elastic Kubernetes Service (EKS).

Terraform simplifies the infrastructure provisioning process. Without Terraform, developers would have to manually configure and provision infrastructure resources for Kubernetes clusters on AWS.

Manual processes are time-consuming, error-prone, and inconsistent across production, staging, and testing environments. Also, they hinder the consistency and repeatability of deployments.

What Are Kubernetes Clusters?

Kubernetes clusters are basically a group of computers that run containerized applications. They are inevitable when you use Kubernetes, which consists of a control plane and multiple compute machines or nodes. Clusters help you monitor the containers to check their status, and once a container goes down, it immediately relaunches, making it fault-tolerant.

What Does AWS EKS Offer?

Elastic Kubernetes Service is a managed service offered by AWS, making the Kubernetes cluster management easier to implement and scale. In other words, if you’re using AWS EKS, you are no longer required to install, operate, and maintain your own Kubernetes control plane on AWS.

How Terraform Comes Into The Picture?

Setting up AWS EKS involves a lot of steps. It may take away many developers’ time if done manually every time. With Terraform, you can write the script once and automate the configuration of the required infrastructure, depending on your application’s requirements. 

Simply put, Terraform provides and manages the infrastructure required to run an EKS cluster. Some of the resources include VPCs, subnets, IAM roles, security groups, and EC2 instances that are necessary for setting up an EKS cluster.

Here's how Terraform collaborates with AWS EKS for infrastructure provisioning and management:

1. VPC And Networking: Terraform specifies the networking components required for an EKS cluster, such as VPCs, subnets, route tables, internet gateways, and NAT gateways. These resources provide the EKS cluster's network infrastructure and associated components.
   
2. IAM Roles And Policies: You can define IAM roles and policies required for EKS, such as roles for the EKS service and worker nodes, enabling access to AWS resources.
   
3. Security Groups: It creates security groups to control inbound and outbound traffic to the EKS cluster and its components, ensuring that only authorized traffic is allowed.
   
4. Auto Scaling Groups: You can define auto-scaling groups for worker nodes in the EKS cluster, allowing for automatic scaling based on resource demand.
   
5. Load Balancers: Terraform provisions and configures load balancers, such as Application or Network Load Balancers, to distribute traffic to the EKS cluster's applications.
   
6. EKS Cluster Configuration: While Terraform itself doesn't manage the EKS cluster directly, it can be used to define data sources or external data that provide inputs to the EKS cluster configuration, such as the Kubernetes version, instance types for worker nodes and other cluster settings.

How To Create EKS Clusters Using Terraform

Follow these steps to provision Kubernetes clusters on AWS using Terraform and EKS:

1. Install: Download and install Terraform on your local machine or CI/CD environment.
2. Define: Write Terraform configuration files (.tf) to define Kubernetes clusters, node groups, networking, security, and other resources.
3. Initialize: Run ‘terraform init’ to initialize the Terraform project and download provider plugins.
4. Plan And Apply: Run ‘terraform plan’ to preview the changes and ‘terraform apply’ to provision the infrastructure on AWS.
5. Deploy Apps: Once the Kubernetes cluster is provisioned, deploy containerized applications using tools like ‘kubectl’ or AWS CodeDeploy.

The below configuration sets up a highly available EKS cluster with managed node groups, VPC networking, and additional Kubernetes add-ons for enhanced functionality. It uses modules for VPC and EKS, making the infrastructure reusable and modular.

Step 1: Install Terraform

Follow the installation instructions for your operating system. For example, on macOS, you can use Homebrew:

On Windows, you can download the binary, extract it, and add it to your PATH.

Verify the Installation:

After installation, verify that Terraform is installed correctly by running:

Step 2: Define

Terraform uses configuration files (.tf) to define the desired state of your infrastructure. For provisioning Kubernetes clusters on AWS, you will need to create several configuration files to define various resources like VPCs, EKS clusters, node groups, networking, and security settings. Here's a breakdown of the essential configuration files:

1 a. terraform.tf: Defines the required providers and the backend configuration.

This:

• Specifies required providers with their versions.
• Configures Terraform to use an S3 bucket for remote state storage.
  • `source = "hashicorp/aws": Indicates that the AWS provider should be downloaded from the HashiCorp registry.
  • `version = "~> 5.47.0": Specifies that the AWS provider version should be 5.47.0 or any minor patch version that is compatible with it (e.g., 5.47.1, but not 5.48.0).
  
  
  Fig. 1. Configured S3 Buckets for Terraform Storage
• Sets the required Terraform version

Fig. 2. Terraform Providers

1 b. variables.tf: Defines the input variables for your configuration.

• This file defines a variable for the AWS region with a default value of "us-west-2".

1 c. main.tf: Contains the main configuration for provisioning the EKS cluster and associated resources.

• This specifies the AWS provider with the region set from a variable

• This fetches the list of available AWS Availability Zones that do not require opt-in.

• This defines the local variables for the VPC name, cluster name, CIDR block for the VPC, and the first three availability zones.
• This also defines the tags for resources.

This:

• Configures a VPC with public and private subnets across three availability zones.
• Enables a single NAT gateway.
• Tags the subnets for Kubernetes ELB and internal ELB roles.

This:

• Creates an EKS cluster with a specified version and public endpoint access.
• Sets up managed node groups with specified instance types and scaling configurations.
• Uses the VPC and subnets created in the VPC module.

This:

• Adds common EKS addons like EBS CSI driver, CoreDNS, VPC CNI, and kube-proxy.
• Enables additional tools like Karpenter (an open-source autoscaler), Prometheus stack, and metrics server.
• Tags the addons with the "dev" environment.

• This configures the Kubernetes provider to interact with the EKS cluster using AWS authentication.

• This configures the Helm provider to manage Kubernetes applications using the Helm package manager.

1 d. output.tf

• This defines outputs for the EKS cluster endpoint, security group ID, region, and cluster name to be easily accessible.

Step 3: Initialize The Project

Navigate to the directory containing your Terraform configuration files and run:

This command will:

• Download and install the necessary provider plugins.
• Configure the backend specified in the terraform.tf file.
• Prepare the working directory for other Terraform commands.

Fig. 3. Output of "terraform init"

Step 4: Plan And Apply

Run ‘terraform plan’ and ‘terraform apply’

• Plan the Infrastructure:

The ‘terraform plan’ command creates an execution plan, showing the changes that will be made to your infrastructure. This step helps you understand what Terraform will do before actually making any changes.

The command will output a detailed list of actions Terraform will perform, such as creating new resources or updating existing ones. Review this output carefully to ensure it matches your expectations.

Fig. 4. Processing "terraform plan"

Fig. 5. Output of "terraform plan"

• Apply the Configuration:

The ‘terraform apply’ command applies the changes required to reach the desired state of the configuration. Terraform will prompt you for confirmation before making any changes.

Confirm the apply action by typing yes when prompted. Terraform will then proceed to create the resources as defined in your configuration files. Depending on the complexity of your infrastructure, this process may take several minutes.

Fig. 6. Output of "terraform apply"

Step 5: Deploy Applications

Once your Kubernetes cluster is provisioned, you can deploy containerized applications using tools like kubectl or AWS CodeDeploy.

• Configure kubectl:

Set up your ‘kubeconfig’ to access the EKS cluster. Terraform provides the cluster endpoint as an output, which you can use to configure kubectl.

Fig. 7. Output of "configure kubectl"

With ‘kubectl’ configured, you can now deploy applications to your Kubernetes cluster. For example, to deploy a simple Nginx application:

Fig. 8. Output of "kubectl create deployment nginx"

Fig. 9. Output of "kubectl expose deployment nginx"

• Monitor Your Applications:

Use ‘kubectl’ commands to monitor the status of your applications and resources. For example:

Fig. 10. Output of "terraform apply"Fig. 11. Output of "terraform apply"

• Automate Deployments with AWS CodeDeploy:

AWS CodeDeploy is a deployment service that automates application deployments to Amazon EC2 instances, on-premises instances, or Lambda functions. To deploy applications using AWS CodeDeploy, create an application and a deployment group, then configure your application’s deployment process using the CodeDeploy console or AWS CLI.

Conclusion

Provisioning Kubernetes clusters on AWS with Terraform and EKS offers a streamlined and efficient approach to deploying containerized applications in the cloud.

Using Terraform's infrastructure-as-code capabilities and EKS's managed Kubernetes service, businesses can automate the provisioning process, improve scalability, enhance security, and accelerate time-to-market for their applications.

Our expert digital engineers empower organizations to embrace cloud-native technologies and drive innovation at scale. Schedule a meeting with them to discuss your niche business requirements.