Welcome to Amazon EKS Blueprints!

• [X] Yes, I've searched similar issues on GitHub and didn't find any.

Amazon EKS Blueprints Release version

4.0.7

What is your environment, configuration and the example used?

terraform version - 1.0.11 awscli version - 2.4.22 error:

Error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

module "eks_blueprints" {
  source                          = "github.com/aws-ia/terraform-aws-eks-blueprints?ref=v4.0.7"
  vpc_id                          = var.eks_vpc_id
  private_subnet_ids              = var.eks_vpc_control_plane
  cluster_version                 = var.cluster_version
  cluster_name                    = local.cluster_name
  # environment                     = var.environment
  # zone                            = var.region
  cluster_endpoint_public_access  = var.cluster_endpoint_public_access
  cluster_endpoint_private_access = var.cluster_endpoint_private_access

  # EKS MANAGED NODE GROUPS
  managed_node_groups = {
    managed_ng_cpu = {
      node_group_name = var.cpu_node_group_name
      ami_type        = var.node_group_ami_type
      instance_types  = var.cpu_instance_types
      min_size        = var.cpu_min_size
      max_size        = var.cpu_max_size
      subnet_ids      = var.eks_vpc_private_subnets_ids
      k8s_labels = {
        NodeType = "CPU"
      }
    },
    managed_ng_gpu = {
      node_group_name = var.gpu_node_group_name
      ami_type        = var.node_group_ami_type
      instance_types  = var.gpu_instance_types
      min_size        = var.gpu_min_size
      max_size        = var.gpu_max_size
      subnet_ids      = var.eks_vpc_private_subnets_ids
      k8s_labels = {
        NodeType = "GPU"
      }
    }
  }
}

module "eks_blueprints_kubernetes_addons" {
  source             = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons?ref=v4.0.7"
  depends_on         = [module.eks_blueprints.managed_node_groups]
  eks_cluster_id     = module.eks_blueprints.eks_cluster_id
  eks_cluster_domain = var.eks_cluster_domain

  # EKS Managed Add-ons
  enable_amazon_eks_vpc_cni    = var.enable_amazon_eks_vpc_cni
  enable_amazon_eks_coredns    = var.enable_amazon_eks_coredns
  enable_amazon_eks_kube_proxy = var.enable_amazon_eks_kube_proxy

  #K8s Add-ons
  enable_aws_load_balancer_controller = var.enable_aws_load_balancer_controller
  enable_metrics_server               = var.enable_metrics_server
  enable_cluster_autoscaler           = var.enable_cluster_autoscaler
  enable_external_dns                 = var.enable_external_dns
  enable_keda                         = var.enable_keda
  enable_crossplane                   = var.enable_crossplane
  enable_aws_for_fluentbit            = var.enable_fluentbit
  enable_cert_manager                 = var.enable_cert_manager
  cert_manager_letsencrypt_email      = local.cert_manager_letsencrypt_email
  cert_manager_domain_names           = [var.eks_cluster_domain]
  enable_argocd                       = var.enable_argocd
  argocd_helm_config = {
    values = [templatefile("${path.module}/helm_values/argocd-values.yaml", {
      ingress_host = local.argocd_host
    })]
  }
  enable_argo_rollouts = var.enable_argo_rollouts
  crossplane_aws_provider = {
    enable                   = var.enable_crossplane
    provider_aws_version     = "v0.24.1"
    additional_irsa_policies = ["arn:aws:iam::aws:policy/AdministratorAccess"]
  }
  aws_for_fluentbit_helm_config = {
    values = [templatefile("${path.module}/helm_values/aws-for-fluentbit-values.yaml", {
      logstash_endpoint = var.logstash_endpoint,
      logstash_port     = var.logstash_port
    })]
  }
}

Provider configuration:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.9"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = ">= 2.10"
    }
    helm = {
      source  = "hashicorp/helm"
      version = ">= 2.4.1"
    }
    kubectl = {
      source  = "gavinbunney/kubectl"
      version = ">= 1.14.0"
    }
  }
}

provider "kubernetes" {
  host                   = module.eks_blueprints.eks_cluster_endpoint
  cluster_ca_certificate = base64decode(module.eks_blueprints.eks_cluster_certificate_authority_data)

  exec {
    api_version = "client.authentication.k8s.io/v1alpha1"
    command     = "aws"
    # This requires the awscli to be installed locally where Terraform is executed
    args = ["eks", "get-token", "--cluster-name", module.eks_blueprints.eks_cluster_id]
  }
}

provider "helm" {
  kubernetes {
    host                   = module.eks_blueprints.eks_cluster_endpoint
    cluster_ca_certificate = base64decode(module.eks_blueprints.eks_cluster_certificate_authority_data)

    exec {
      api_version = "client.authentication.k8s.io/v1alpha1"
      command     = "aws"
      # This requires the awscli to be installed locally where Terraform is executed
      args = ["eks", "get-token", "--cluster-name", module.eks_blueprints.eks_cluster_id]
    }
  }
}

provider "kubectl" {
  apply_retry_count      = 10
  host                   = module.eks_blueprints.eks_cluster_endpoint
  cluster_ca_certificate = base64decode(module.eks_blueprints.eks_cluster_certificate_authority_data)
  load_config_file       = false

  exec {
    api_version = "client.authentication.k8s.io/v1alpha1"
    command     = "aws"
    # This requires the awscli to be installed locally where Terraform is executed
    args = ["eks", "get-token", "--cluster-name", module.eks_blueprints.eks_cluster_id]
  }
}

What did you do and What did you see instead?

Tried updating module from 4.0.6 to 4.0.7 and getting connection error while trying t install helm chart:

Error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

reverting back to 4.0.6 helps

Additional Information

tried "jumping" to 4.0.9 with the outputs change:

eks_oidc_provider = module.eks_blueprints.oidc_provider
eks_cluster_endpoint = module.eks_blueprints.eks_cluster_endpoint
eks_cluster_version  = module.eks_blueprints.eks_cluster_version

still facing the same issue

Try something like this.

data "aws_eks_cluster" "cluster" {
  name = module.eks_blueprints.eks_cluster_id
}

data "aws_eks_cluster_auth" "cluster" {
  name = module.eks_blueprints.eks_cluster_id
}

provider "kubernetes" {
  experiments {
    manifest_resource = true
  }
  host                   = data.aws_eks_cluster.cluster.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)
  token                  = data.aws_eks_cluster_auth.cluster.token
}

Hey, i do have these data blocks:

data "aws_availability_zones" "available" {}

data "aws_eks_cluster" "cluster" {
  name = module.eks_blueprints.eks_cluster_id
}

data "aws_eks_cluster_auth" "cluster" {
  name = module.eks_blueprints.eks_cluster_id
}

In addition, I tried configuring providers that way and it didn't work as well, I got the same error

From my experience seeing these type of issues, it's something to do with the providers config. I recently saw it when helm provider was missing from an example, but in your case I can see the helm provider is setup.

I believe one of the changes that may affect this is moving away from requiring env/zone/tenant (4.0.6) which defines the cluster name, and with 4.0.7 you just need to pass the cluster name.

Can you please double check that the cluster name matching the cluster name that was defined and created originally with 4.0.6, you can do that by checking the state file itself or simply checking the name of the EKS cluster.

closing out the issue for now - please try upgrading to the latest version and let us know if your issue still persists. thank you