Using nested variable blocks from a configmap/secret

[Smana]

Hi,

I'm trying to use a module that requires variables with maps, lists ... Let's say that I want to use this irsa-eks module. How would I define the variables into a configmap please? The following code won't work obviously

apiVersion: v1
data:
  role_name: "cert-manager"
  attach_load_balancer_controller_policy: true
  oidc_providers:
    main:
      provider_arn: ${oidc_cluster_issuer}
      namespace_service_accounts: ["kube-system:alb-controller"]
  tags:
    env: ${environment}
    cluster: ${cluster_name}
kind: ConfigMap
metadata:
  name: tfvars-dev-hiving-hen
  namespace: flux-system

Thank you

========

User Story:

As a Terraform Controller user, I'd like to be able to use nested variable blocks from a configmap/secret, so that I can manage complex variables and sensitive data in a secure and organized manner.

Acceptance Criteria:

• [ ] Support for using nested variables from ConfigMaps
• [ ] Support for using nested variables from Secrets
• [ ] Documentation on how to use nested variables from ConfigMaps and Secrets
• [ ] Examples of using nested variables from ConfigMaps and Secrets

[chanwit]

Thank you for bringing this up @Smana

You're right. We currently don't have a mechanism to serialize complex structure like above into a variable. I mean we do have this capability but for somewhere else, which is spec.values. And you are required to declare the values variable in your Terraform.

The main reason we don't support this in varsFrom because it requires you to supply "type information".

[chanwit]

I'll check to see if we would find some good tricks to deserialize structured data as HCL's map.

[Smana]

Hi @chanwit , thanks for your answer. Indeed using vars works like charm. Here is an example

---
apiVersion: infra.contrib.fluxcd.io/v1alpha2
kind: Terraform
metadata:
  name: irsa-alb-controller
  namespace: flux-system
spec:
  approvePlan: auto
  destroyResourcesOnDeletion: true
  interval: 8m
  path: ./modules/iam-role-for-service-accounts-eks
  sourceRef:
    kind: GitRepository
    name: terraform-aws-iam
    namespace: flux-system
  vars:
    - name: role_name
      value: ${cluster_name}-load-balancer-controller
    - name: attach_load_balancer_controller_policy
      value: true
    - name: oidc_providers
      value:
        main:
          provider_arn: "arn:aws:iam::xxx:oidc-provider/oidc.eks.eu-west-3.amazonaws.com/id/xxx"
          namespace_service_accounts: ["kube-system:alb-controller"]
---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: terraform-aws-iam
  namespace: flux-system
spec:
  interval: 30s
  ref:
    branch: master
  url: https://github.com/terraform-aws-modules/terraform-aws-iam

However that would be very useful to source the vars from secrets for example. If the sensitive data is in a nested map.

[shencan]

Any updates for this issue, varsFrom only support configmap value is string, not support hcl map or list.

[chanwit]

Hi @shencan

We still don't have a good solution to this problem. A feasible implementation would be allowing a user to manually add type information to secrets or configmaps, but it would cause another UX problem. No one would want to add that extra info by themselves.

Maybe we would try introducing .yaml like this:

kind: ConfigMap
data:
  my-config.yaml: |
    regions:
      - us-west-1
      - us-east-1

Please note that we will not support parsing HCL in Secrets or ConfigMaps, but we might support conversion of YAML or JSON to HCL.

[ilithanos]

currently i've managed to work around this by using json as output and input variables for my terraform modules, and then simply jsonencode on output and jsondecode on input variables within my terraform modules. It's a workaround, but it works pretty well doing it this way.

Could be an option to look into doing this automatically as part of the controller using either yaml or json.

[jkossis]

@chanwit just wanted to re-alive this, is there any movement on this?

This would be a killer feature (especially for secret data in a nested type).