terraform-provider-helm

terraform-provider-helm copied to clipboard

If you prefer, you can also ask your question in the Kubernetes community Slack channel #terraform-providers. (Sign up here)

Terraform version, Kubernetes provider version and Kubernetes version

Terraform version: 1.1.2
Helm Provider version: 2.4.1
Kubernetes version: 2.7.1

Terraform configuration

Enter your configuration here.


provider "helm" {
  kubernetes {
    host = azurerm_kubernetes_cluster.aks.kube_config[0].host
    client_certificate = base64decode(azurerm_kubernetes_cluster.aks.kube_config[0].client_certificate)
    client_key = base64decode(azurerm_kubernetes_cluster.aks.kube_config[0].client_key)
    cluster_ca_certificate = base64decode(azurerm_kubernetes_cluster.aks.kube_config[0].cluster_ca_certificate)
    
  }
  debug = true
}


resource "helm_release" "nginx_ingress" {
  name = "ingress-nginx"
  repository = "https://kubernetes.github.io/ingress-nginx"  
  chart = "ingress-nginx"  
  version ="4.0.6"
  namespace = "ingress"  
  create_namespace = true
  timeout = 3600
  wait = true

 values = [

    "${file("./pre/private-ingress.yaml")}"

  ]

  set{
    name="rbac.create"
    value=true
  }

  set{
    name="controller.service.annotations.service.beta.kubernetes.io/azure-load-balancer-internal"
    value="true"
  }

  set{
    name="controller.service.annotations.service.beta.kubernetes.io/azure-load-balancer-internal-subnet"
    value="snet-ilb-aks-pre-condorcloud"
  }



  depends_on = [
    azurerm_kubernetes_cluster.aks
  ]
  
}

---------------------------------------------------------------------------
private-ingress.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx
  name: ingress-nginx-controller  
  namespace: ingress
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    service.beta.kubernetes.io/azure-load-balancer-internal-subnet: "snet-ilb-aks-pre-condorcloud"
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer
  loadBalancerIP: 10.80.181.132

--------------------------------------------------------------------------------

Terraform console output 

# helm_release.nginx_ingress will be created
  + resource "helm_release" "nginx_ingress" {
      + atomic                     = false
      + chart                      = "ingress-nginx"
      + cleanup_on_fail            = false
      + create_namespace           = true
      + dependency_update          = false
      + disable_crd_hooks          = false
      + disable_openapi_validation = false
      + disable_webhooks           = false
      + force_update               = false
      + id                         = (known after apply)
      + lint                       = false
      + manifest                   = (known after apply)
      + max_history                = 0
      + metadata                   = (known after apply)
      + name                       = "ingress-nginx"
      + namespace                  = "ingress"
      + recreate_pods              = false
      + render_subchart_notes      = true
      + replace                    = false
      + repository                 = "https://kubernetes.github.io/ingress-nginx"
      + reset_values               = false
      + reuse_values               = false
      + skip_crds                  = false
      + status                     = "deployed"
      + timeout                    = 3600
      + values                     = [
          + <<-EOT
                apiVersion: v1
                kind: Service
                metadata:
                  labels:
                    app: nginx
                  name: ingress-nginx-controller
                  namespace: ingress
                  annotations:
                    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
                    service.beta.kubernetes.io/azure-load-balancer-internal-subnet: "snet-ilb-aks-pre-condorcloud"
                spec:
                  ports:
                  - port: 80
                    protocol: TCP
                    targetPort: 80
                  selector:
                    app: nginx
                  type: LoadBalancer
                  loadBalancerIP: 10.80.181.132
            EOT,
        ]
      + verify                     = false
      + version                    = "4.0.6"
      + wait                       = true
      + wait_for_jobs              = false

      + set {
          + name  = "controller.service.annotations.service.beta.kubernetes.io/azure-load-balancer-internal"
          + value = "true"
        }
      + set {
          + name  = "controller.service.annotations.service.beta.kubernetes.io/azure-load-balancer-internal-subnet"
          + value = "snet-ilb-aks-pre-condorcloud"
        }
      + set {
          + name  = "rbac.create"
          + value = "true"
        }
    }

Question

Enter your question here.

when we configure the helm release to apply an azure internal loab balancer and we indicate the internal subnet in the terraform it appears that it will be deployed correctly with the values provided, but neither with the yaml nor with the sets the configuration is established. 

A load balancer is set with a public ip, this may be a bug or is that the configuration is not correct?

Im seeing the exact same thing, but I'm trying to set annotations w/ the Traefik chart

No matter what variation I use w/ this helm provider, I cannot set any chart values that require an object/map where the "key" contains dots/periods etc such as the OP has listed above.

this is maddening

I've tried a direct values.yaml file include like the poster above does, I've tried basic sets while escaping the periods etc. nothing works.

Not sure if this will solve your issue but with helm_resource sets if you are setting a value within an object you need to escape the extra . in the key. Something like:

  set{
    name="controller.service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-internal"
    value="true"
  }

  set{
    name="controller.service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-internal-subnet"
    value="snet-ilb-aks-pre-condorcloud"
  }

This will maintain the annotation key value correctly - this lets you set values in objects from Terraform.

Marking this issue as stale due to inactivity. If this issue receives no comments in the next 30 days it will automatically be closed. If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. This helps our maintainers find and focus on the active issues. Maintainers may also remove the stale label at their discretion. Thank you!