[FEATURE] Additional helm values for kubernetes-addons which won't delete the default

[dex4er]

Is your feature request related to a problem? Please describe

I can't just add additional values for helm. I have to replace the default valyes.yaml with full copy of all values.

Ie.

module "aws_eks_addons" {
  external_dns_helm_config = {
    values = [yamlencode({
      sources : ["service", "istio-gateway"]
    })]
  }
}

will remove settings from the default values.yaml like provider, zoneIdFilters and aws.region.

Describe the solution you'd like

I would like to see such pattern (example for externa-dns):

variable "helm_values" {
  type        = any
  default     = {}
  description = "Additional Helm values"
}

locals {
  default_helm_config = {
    values      = [local.default_helm_values, var.helm_values]
  }
  default_helm_values = templatefile("${path.module}/values.yaml", {
    aws_region      = var.addon_context.aws_region_name
    zone_filter_ids = local.zone_filter_ids
  })
}

Then:

module "aws_eks_addons" {
  external_dns_helm_values = yamlencode({
    sources : ["service", "istio-gateway"]
  )}
}

The pattern uses ability of Helm which merges 2 or more values.yaml files.

[andrewhibbert]

This is still a valid issue, just uses external-dns as an example. The solution given should be possible for all add-ons

[florentio]

Hello @dex4er. This issue is relevant and I do have a proposal. I would like to share a proposal of a fix here and get some comments if possible before adding a PR

I am going to use karpenter add-on but the proposal is applicable to any add-on on the repository Currently we already have a way to override the helm configuration. here is an example to set a value file to the helm chart

  enable_karpenter = true
  # Optional  karpenter_helm_config
  karpenter_helm_config = {
    name                       = "karpenter"
    chart                      = "karpenter"
    repository                 = "https://charts.karpenter.sh"
    version                    = "0.6.3"
    namespace                  = "karpenter"
    values = [templatefile("${path.module}/values.yaml", {
         eks_cluster_id       = var.eks_cluster_id,
         eks_cluster_endpoint = var.eks_cluster_endpoint,
         service_account_name = var.service_account_name,
         operating_system     = "linux"
    })]
  }

There is a many way to set the values depending on how the consumer is building the raw yaml content. Function like yamlencode, templatefile, file etc... can be used.

Below is an example with 3 different additional values

  karpenter_helm_config = {
    name  = "karpenter"
    values = [yamlencode({ priorityClassName : "system-cluster-critical"}), 
               templatefile("${path.module}/karpenter-another-values.yaml", {}),
               file("karpenter-third-values.yaml")
    ]
  }

The only change to be applied will be on the add-on module side here is how to build helm_config variable to be passed to the helm-add on

locals {
  name                 = "karpenter"
  service_account_name = "karpenter"
  default_helm_config = {
    name        = local.name
    chart       = local.name
    repository  = "https://charts.karpenter.sh"
    version     = "0.16.0"
    namespace   = local.name
    timeout     = "300"
    values      = local.default_helm_values
    set         = []
    description = "karpenter Helm Chart for Node Autoscaling"
  }

  helm_config = merge(
    local.default_helm_config,
    var.helm_config
  )

  default_helm_values = [templatefile("${path.module}/values.yaml", {
    eks_cluster_id            = var.addon_context.eks_cluster_id,
    eks_cluster_endpoint      = local.eks_cluster_endpoint,
    node_iam_instance_profile = var.node_iam_instance_profile,
    operating_system          = "linux"
  })]
}

the new way will be

locals {
  name                 = "karpenter"
  service_account_name = "karpenter"
  default_helm_config = {
    name        = local.name
    chart       = local.name
    repository  = "https://charts.karpenter.sh"
    version     = "0.16.0"
    namespace   = local.name
    timeout     = "300"
    set         = []
    description = "karpenter Helm Chart for Node Autoscaling"
  }

  helm_config = merge(
    local.default_helm_config,
    var.helm_config,
    {values = concat(try(var.helm_config["values"], []), local.default_helm_values)}
  )

  default_helm_values = [templatefile("${path.module}/values.yaml", {
    eks_cluster_id            = var.addon_context.eks_cluster_id,
    eks_cluster_endpoint      = local.eks_cluster_endpoint,
    node_iam_instance_profile = var.node_iam_instance_profile,
    operating_system          = "linux"
  })]
}