Migrate to EKS Module v19.x

Please consult the docs/v4-to-v5/example directory for reference configurations. If you find a bug, please open an issue with supporting configuration to reproduce.

Backwards incompatible changes

• The cluster module provided in EKS Blueprints is being removed entirely from the project. Instead, users are encouraged to use the terraform-aws-eks module for creating and managing their EKS cluster in Terraform.
• The KMS module provided in EKS Blueprints has been removed. Users can leverage the KMS creation/management functionality provided by the terraform-aws-eks module or utilize the standalone terraform-aws-kms module.
• The EMR on EKS module provided in EKS Blueprints has been removed. Instead, users are encouraged to use the terraform-aws-emr virtual cluster sub-module for creating and managing their EMR on EKS virtual cluster in Terraform.
• The teams multi-tenancy module provided in EKS Blueprints has been removed. Instead, users are encouraged to use the terraform-aws-eks-blueprints-teams module for creating and managing their multi-tenancy constructions within their EKS clusters in Terraform.

Additional changes

Added

• N/A

Modified

• N/A

Removed

• All noted above under Backwards incompatible changes

Variable and output changes

Since the change is to replace the EKS Blueprints cluster module with the terraform-aws-eks module, there aren't technically any variable or output changes other than their removal. Please consult the terraform-aws-eks module for its respective variables/outputs.

1. Removed variables:

   • All

2. Renamed variables:

   • None

3. Added variables:

   • None

4. Removed outputs:

   • All

5. Renamed outputs:

   • None

6. Added outputs:

   • None

Upgrade Migrations

Before - v4.32 Example

module "eks" {
  source = "github.com/aws-ia/terraform-aws-eks-blueprints?ref=v4.32.1"

  cluster_name    = local.name
  cluster_version = "1.26"

  vpc_id                          = module.vpc.vpc_id
  private_subnet_ids              = module.vpc.private_subnets
  cluster_endpoint_private_access = true

  map_roles = [
    {
      rolearn  = data.aws_caller_identity.current.arn
      username = "me"
      groups   = ["system:masters"]
    },
  ]

  managed_node_groups = {
    managed = {
      node_group_name = "managed"
      instance_types  = ["m5.large"]

      min_size     = 1
      max_size     = 2
      desired_size = 1

      k8s_labels = {
        Which = "managed"
      }
    }
  }

  fargate_profiles = {
    fargate = {
      fargate_profile_name = "fargate"
      fargate_profile_namespaces = [{
        namespace = "default"

        k8s_labels = {
          Which = "fargate"
        }
      }]
      subnet_ids = module.vpc.private_subnets
    }
  }

  self_managed_node_groups = {
    self_managed = {
      node_group_name    = "self_managed"
      instance_type      = "m5.large"
      launch_template_os = "amazonlinux2eks"

      min_size     = 1
      max_size     = 2
      desired_size = 1

      k8s_labels = {
        Which = "self-managed"
      }
    }
  }

  tags = {
    Blueprint  = local.name
    GithubRepo = "github.com/aws-ia/terraform-aws-eks-blueprints"
  }
}

After - v5.0 Example

Any of the values that are marked with # Backwards compat are provided to demonstrate configuration level changes to reduce the number of Terraform changes when migrating to the EKS module.

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.13"

  cluster_name                   = local.name
  cluster_version                = "1.26"
  cluster_endpoint_public_access = true # Backwards compat
  cluster_enabled_log_types      = ["api", "audit", "authenticator", "controllerManager", "scheduler"] # Backwards compat

  iam_role_name            = "${local.name}-cluster-role" # Backwards compat
  iam_role_use_name_prefix = false                        # Backwards compat

  kms_key_aliases = [local.name] # Backwards compat

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  manage_aws_auth_configmap = true
  aws_auth_roles = [
    {
      rolearn  = data.aws_caller_identity.current.arn
      username = "me"
      groups   = ["system:masters"]
    },
  ]

  eks_managed_node_groups = {
    managed = {
      iam_role_name              = "${local.name}-managed" # Backwards compat
      iam_role_use_name_prefix   = false                   # Backwards compat
      use_custom_launch_template = false                   # Backwards compat

      instance_types = ["m5.large"]

      min_size     = 1
      max_size     = 2
      desired_size = 1

      labels = {
        Which = "managed"
      }
    }
  }

  fargate_profiles = {
    fargate = {
      iam_role_name            = "${local.name}-fargate" # Backwards compat
      iam_role_use_name_prefix = false                   # Backwards compat

      selectors = [{
        namespace = "default"
        labels = {
          Which = "fargate"
        }
      }]
    }
  }

  self_managed_node_groups = {
    self_managed = {
      name            = "${local.name}-self_managed" # Backwards compat
      use_name_prefix = false                        # Backwards compat

      iam_role_name            = "${local.name}-self_managed" # Backwards compat
      iam_role_use_name_prefix = false                        # Backwards compat

      launch_template_name            = "self_managed-${local.name}" # Backwards compat
      launch_template_use_name_prefix = false                        # Backwards compat

      instance_type = "m5.large"

      min_size     = 1
      max_size     = 2
      desired_size = 1

      labels = {
        Which = "self-managed"
      }
    }
  }

  tags = {
    Blueprint  = local.name
    GithubRepo = "github.com/aws-ia/terraform-aws-eks-blueprints"
  }
}

Diff of Before vs After

module "eks" {
-  source = "github.com/aws-ia/terraform-aws-eks-blueprints?ref=v4.32.1"
+  source  = "terraform-aws-modules/eks/aws"
+  version = "~> 19.13"

  cluster_name    = local.name
  cluster_version = "1.26"

  vpc_id                          = module.vpc.vpc_id
  private_subnet_ids              = module.vpc.private_subnets
+  cluster_endpoint_public_access  = true
-  cluster_endpoint_private_access = true

-  map_roles = [
+  aws_auth_roles = [
    {
      rolearn  = data.aws_caller_identity.current.arn
      username = "me"
      groups   = ["system:masters"]
    },
  ]

-  managed_node_groups = {
+  eks_managed_node_groups = {
    managed = {
-      node_group_name = "managed"
      instance_types  = ["m5.large"]

      min_size     = 1
      max_size     = 2
      desired_size = 1

-      k8s_labels = {
+      labels = {
        Which = "managed"
      }
    }
  }

  fargate_profiles = {
    fargate = {
-      fargate_profile_name = "fargate"
-      fargate_profile_namespaces = [{
+      selectors = [{
        namespace = "default"

-        k8s_labels = {
+        labels = {
          Which = "fargate"
        }
      }]
-      subnet_ids = module.vpc.private_subnets
    }
  }

  self_managed_node_groups = {
    self_managed = {
-      node_group_name    = "self_managed"
      instance_type      = "m5.large"
-      launch_template_os = "amazonlinux2eks"

      min_size     = 1
      max_size     = 2
      desired_size = 1

-      k8s_labels = {
+      labels = {
        Which = "self-managed"
      }
    }
  }

  tags = {
    Blueprint  = local.name
    GithubRepo = "github.com/aws-ia/terraform-aws-eks-blueprints"
  }
}

State Move Commands

The following Terraform state move commands are provided to aid in migrating the control plane and data plane components.

# This is not removing the configmap from the cluster -
# it will be adopted by the new module
terraform state rm 'module.eks.kubernetes_config_map.aws_auth[0]'

# Cluster
terraform state mv 'module.eks.module.aws_eks.aws_eks_cluster.this[0]' 'module.eks.aws_eks_cluster.this[0]'

# Cluster IAM role
terraform state mv 'module.eks.module.aws_eks.aws_iam_role.this[0]' 'module.eks.aws_iam_role.this[0]'
terraform state mv 'module.eks.module.aws_eks.aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"]' 'module.eks.aws_iam_role_policy_attachment.this["AmazonEKSClusterPolicy"]'
terraform state mv 'module.eks.module.aws_eks.aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"]' 'module.eks.aws_iam_role_policy_attachment.this["AmazonEKSVPCResourceController"]'

# Cluster primary security group tags
# Note: This will depend on the tags applied to the module - here we
#       are demonstrating the two tags used in the configuration above
terraform state mv 'module.eks.module.aws_eks.aws_ec2_tag.cluster_primary_security_group["Blueprint"]' 'module.eks.aws_ec2_tag.cluster_primary_security_group["Blueprint"]'
terraform state mv 'module.eks.module.aws_eks.aws_ec2_tag.cluster_primary_security_group["GithubRepo"]' 'module.eks.aws_ec2_tag.cluster_primary_security_group["GithubRepo"]'

# Cluster security group
terraform state mv 'module.eks.module.aws_eks.aws_security_group.cluster[0]' 'module.eks.aws_security_group.cluster[0]'

# Cluster security group rules
terraform state mv 'module.eks.module.aws_eks.aws_security_group_rule.cluster["ingress_nodes_443"]' 'module.eks.aws_security_group_rule.cluster["ingress_nodes_443"]'

# Node security group
terraform state mv 'module.eks.module.aws_eks.aws_security_group.node[0]' 'module.eks.aws_security_group.node[0]'

# Node security group rules
terraform state mv 'module.eks.module.aws_eks.aws_security_group_rule.node["ingress_cluster_443"]' 'module.eks.aws_security_group_rule.node["ingress_cluster_443"]'
terraform state mv 'module.eks.module.aws_eks.aws_security_group_rule.node["ingress_cluster_kubelet"]' 'module.eks.aws_security_group_rule.node["ingress_cluster_kubelet"]'
terraform state mv 'module.eks.module.aws_eks.aws_security_group_rule.node["ingress_self_coredns_tcp"]' 'module.eks.aws_security_group_rule.node["ingress_self_coredns_tcp"]'
terraform state mv 'module.eks.module.aws_eks.aws_security_group_rule.node["ingress_self_coredns_udp"]' 'module.eks.aws_security_group_rule.node["ingress_self_coredns_udp"]'

# OIDC provider
terraform state mv 'module.eks.module.aws_eks.aws_iam_openid_connect_provider.oidc_provider[0]' 'module.eks.aws_iam_openid_connect_provider.oidc_provider[0]'

# Fargate profile(s)
# Note: This demonstrates migrating one profile that is stored under the
#       key `fargate` in the module definition. The same set of steps would
#       need to be performed for each profile, changing only the key name
terraform state mv 'module.eks.module.aws_eks_fargate_profiles["fargate"].aws_eks_fargate_profile.eks_fargate' 'module.eks.module.fargate_profile["fargate"].aws_eks_fargate_profile.this[0]'
terraform state mv 'module.eks.module.aws_eks_fargate_profiles["fargate"].aws_iam_role.fargate[0]' 'module.eks.module.fargate_profile["fargate"].aws_iam_role.this[0]'
terraform state mv 'module.eks.module.aws_eks_fargate_profiles["fargate"].aws_iam_role_policy_attachment.fargate_pod_execution_role_policy["arn:aws:iam::aws:policy/AmazonEKSFargatePodExecutionRolePolicy"]' 'module.eks.module.fargate_profile["fargate"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKSFargatePodExecutionRolePolicy"]'

# Managed nodegroup(s)
# Note: This demonstrates migrating one nodegroup that is stored under the
#       key `managed` in the module definition. The same set of steps would
#       need to be performed for each nodegroup, changing only the key name
terraform state mv 'module.eks.module.aws_eks_managed_node_groups["managed"].aws_eks_node_group.managed_ng' 'module.eks.module.eks_managed_node_group["managed"].aws_eks_node_group.this[0]'
terraform state mv 'module.eks.module.aws_eks_managed_node_groups["managed"].aws_iam_role.managed_ng[0]' 'module.eks.module.eks_managed_node_group["managed"].aws_iam_role.this[0]'
terraform state mv 'module.eks.module.aws_eks_managed_node_groups["managed"].aws_iam_role_policy_attachment.managed_ng["arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"]' 'module.eks.module.eks_managed_node_group["managed"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"]'
terraform state mv 'module.eks.module.aws_eks_managed_node_groups["managed"].aws_iam_role_policy_attachment.managed_ng["arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"]' 'module.eks.module.eks_managed_node_group["managed"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"]'
terraform state mv 'module.eks.module.aws_eks_managed_node_groups["managed"].aws_iam_role_policy_attachment.managed_ng["arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"]' 'module.eks.module.eks_managed_node_group["managed"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"]'

# Self-managed nodegroup(s)
# Note: This demonstrates migrating one nodegroup that is stored under the
#       key `self_managed` in the module definition. The same set of steps would
#       need to be performed for each nodegroup, changing only the key name
terraform state mv 'module.eks.module.aws_eks_self_managed_node_groups["self_managed"].aws_autoscaling_group.self_managed_ng' 'module.eks.module.self_managed_node_group["self_managed"].aws_autoscaling_group.this[0]'
terraform state mv 'module.eks.module.aws_eks_self_managed_node_groups["self_managed"].aws_iam_instance_profile.self_managed_ng[0]' 'module.eks.module.self_managed_node_group["self_managed"].aws_iam_instance_profile.this[0]'
terraform state mv 'module.eks.module.aws_eks_self_managed_node_groups["self_managed"].aws_iam_role.self_managed_ng[0]' 'module.eks.module.self_managed_node_group["self_managed"].aws_iam_role.this[0]'
terraform state mv 'module.eks.module.aws_eks_self_managed_node_groups["self_managed"].aws_iam_role_policy_attachment.self_managed_ng["arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"]' 'module.eks.module.self_managed_node_group["self_managed"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"]'
terraform state mv 'module.eks.module.aws_eks_self_managed_node_groups["self_managed"].aws_iam_role_policy_attachment.self_managed_ng["arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"]' 'module.eks.module.self_managed_node_group["self_managed"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"]'
terraform state mv 'module.eks.module.aws_eks_self_managed_node_groups["self_managed"].aws_iam_role_policy_attachment.self_managed_ng["arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"]' 'module.eks.module.self_managed_node_group["self_managed"].aws_iam_role_policy_attachment.this["arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"]'
terraform state mv 'module.eks.module.aws_eks_self_managed_node_groups["self_managed"].module.launch_template_self_managed_ng.aws_launch_template.this["self-managed-node-group"]' 'module.eks.module.self_managed_node_group["self_managed"].aws_launch_template.this[0]'

# Secrets KMS key
terraform state mv ' module.eks.module.kms[0].aws_kms_key.this' 'module.eks.module.kms.aws_kms_key.this[0]'
terraform state mv 'module.eks.module.kms[0].aws_kms_alias.this' 'module.eks.module.kms.aws_kms_alias.this["migration"]'

# Cloudwatch Log Group
terraform import 'module.eks.aws_cloudwatch_log_group.this[0]' /aws/eks/migration/cluster

Removed Resources

The following resources will be destroyed when migrating from EKS Blueprints v4.32.1 cluster to the v19.x EKS cluster:

module.eks.module.aws_eks_managed_node_groups["managed"].aws_iam_instance_profile.managed_ng[0]

• It is not directly used and was intended to be used by Karpenter. The https://github.com/aws-ia/terraform-aws-eks-blueprints-addons module provides its own resource for creating an IAM instance profile for Karpenter

module.eks.module.aws_eks_managed_node_groups["managed"].aws_iam_role_policy_attachment.managed_ng["arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"]

• IAM policy is not required by EKS - users can re-add this policy at their discretion

module.eks.module.aws_eks_fargate_profiles["fargate"].aws_iam_policy.cwlogs[0]
module.eks.module.aws_eks_fargate_profiles["fargate"].aws_iam_role_policy_attachment.cwlogs[0]

• Policy is not required by EKS

module.eks.module.aws_eks_self_managed_node_groups["self_managed"].aws_iam_role_policy_attachment.self_managed_ng["arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"]

• IAM policy is not required by EKS - users can re-add this policy at their discretion