Existing Namespace Error Not Recognized by GitOps

[cajual]

Description

If you have a namespace in your EKS Blueprint cluster that is not within the Terraform state and you attempt to use GitOps Add-ons which touch that Namespace, two issues occur:

1. The Namespace will be created by the module add-on before being handed off to the chart repository;
2. Existing namespace will be rejected, but defining a helm_config will fail the GitOps deployment.

https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/21b9da84cce0000d5f78cbeeaca084b928288a2f/modules/kubernetes-addons/prometheus/main.tf#L68

This issue was encountered when switching from a "workload" prometheus to an "add-on" prometheus, using the same Namespace. I attempted to remedy this through Argo CD and the Helm Chart, but the error came from Line 68 of the above manifest. Thus, even Argo managed add-ons require some sort of try or helm_config variability in the instance a namespace already exists.

This issue was remedied by manually deleting the namespace, but that was because I had no other resources in it. A namespace with existing resources could post a problem.

• [x] ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version [Required]: 4.8.1

• Terraform version: Terraform v1.2.7

• Provider version(s):

• provider registry.terraform.io/gavinbunney/kubectl v1.14.0
• provider registry.terraform.io/hashicorp/aws v4.27.0
• provider registry.terraform.io/hashicorp/cloudinit v2.2.0
• provider registry.terraform.io/hashicorp/helm v2.6.0
• provider registry.terraform.io/hashicorp/kubernetes v2.13.0
• provider registry.terraform.io/hashicorp/local v2.2.3
• provider registry.terraform.io/hashicorp/null v3.1.1
• provider registry.terraform.io/hashicorp/random v3.3.2
• provider registry.terraform.io/hashicorp/time v0.8.0
• provider registry.terraform.io/hashicorp/tls v3.4.0
• provider registry.terraform.io/terraform-aws-modules/http v2.4.1

Reproduction Code [Required]

enable_prometheus = true
prometheus_helm_config = {
    name = "prometheus"
    repository = "https://prometheus-community.github.io/helm-charts"
    chart = "prometheus"
    namespace  = "monitoring" # changed from "prometheus" due to conflict, but GitOps fails to deploy
}

Steps to reproduce the behavior:

Deploy Promtail as a workload or stand-alone deployment into the Prometheus namespace, then try to enable the Prometheus add-on.

Expected behaviour

Create if not exist.

Actual behaviour

Error, namespace exists.

Terminal Output Screenshot(s)

Error: Namespace "prometheus" already exists.

[florentio]

Hello @gibbonsryan If the namespace already exists, you have to set the create_namespace attribute to false for your config like this

enable_prometheus = true
prometheus_helm_config = {
    name = "prometheus"
    repository = "https://prometheus-community.github.io/helm-charts"
    chart = "prometheus"
    create_namespace = false
    namespace  = "prometheus"
}

[bryantbiggs]

thank you @florentio, thats correct. closing with guidance provided above