terraform-aws-eks-jx
terraform-aws-eks-jx copied to clipboard
jenkins-x
kuberhealthy installation uses outdated cluster URL
Summary
When creating a new EKS cluster from scratch with terraform, kuberhealthy is installed with the cluster URL from the previous run of terraform.
Steps to reproduce the behavior
terraform apply
# Clean up ELB
terraform destroy
# Check if everything is cleaned up
terraform apply
Expected behavior
Cluster is created and kuberhealthy is installed.
Actual behavior
Error: Kubernetes cluster unreachable: Get https://<old url>.gr7.us-east-1.eks.amazonaws.com/version?timeout=32s: dial tcp: lookup <old url>.gr7.us-east-1.eks.amazonaws.com on [2a02:908:2:a::1]:53: no such host
on .terraform/modules/eks-jx.health.jx-health/main.tf line 1, in resource "helm_release" "kuberhealthy":
1: resource "helm_release" "kuberhealthy" {
where <old url> is the cluster URL of the previous run. Note that kubeconfig is correctly updated with the new cluster when terraform aborts with the error message.
Maybe there is a missing dependency or race condition. The terraform log shows first
module.eks-jx.module.health.module.jx-health[0].helm_release.kuberhealthy: Creating...
and then much later
module.eks-jx.module.cluster.module.eks.local_file.kubeconfig[0]: Creating...
module.eks-jx.module.cluster.module.eks.local_file.kubeconfig[0]: Creation complete after 0s [id=294be563e46ce5a5d60a5d34b2737b7c55791cda]
Terraform version
The output of terraform version is:
Terraform v0.13.5
Module version
1.11.0
Operating system
MacOS
Workaround is probably to remove the previous kubeconfig entry after terraform destroy. I will try it out next time and add a comment to the cleanup procedure.
@eickler, @ankitm123 I have exactly the same problem, shall I delete the content of ~/.kube folder after terraform destroy?
The latest version should fix that issue - the issue was that the helm provider was using outdated credentials. It should work now in the latest version, can someone try and confirm?
