[bug] git generator wont add map or list as values

[ofirshtrull]

Hello to all I want to have an applicationset for an external chart i.e kube-prometheus-stack each cluster has many different values and I want to manage the values via a clean and per env value file so I created the next two files config/prometheus-operator/dev.yaml

cluster_name: test-cluster
cluster_url: "https://8.8.8.8/"
cluster_env: "test"

prometheus:
  ofir:
    i_work_to: yeap

prometheus-applicationset.yaml

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: kube-prometheus-stack
  namespace: argocd
spec:
  generators:
  - git:
      repoURL: https://github.com/xxxx/argocd-config.git
      revision: infra
      files:
      - path: "config/prometheus-operator/*.yaml"
  template:
    metadata:
      labels:
        cluster: "{{cluster_name}}"
      name: "prometheus-stack-{{cluster_name}}"
      finalizers:
        - resources-finalizer.argocd.argoproj.io
    spec:
      project: default
      source:
        chart: kube-prometheus-stack
        repoURL: https://prometheus-community.github.io/helm-charts
        targetRevision: 33.2.0
        helm:
          values: |
            nameOverride: kube-prometheus-stack
            '{{prometheus}}' # result is a map


      destination:
        server: "{{ cluster_url }}"
        namespace: "monitoring"
      syncPolicy:
        automated:
          selfHeal: true
          prune: true
        syncOptions:
          - CreateNamespace=true

the main problem I get is that I cant add values that are set as a map any workaround? or trick to get them to work? and just an FYI ofir: '{{prometheus.ofir.i_work_to}}' # result is a string work just fine, the problem starts when I try to pass a map

also I don't care if I also use the full file as values but I need something to work :-)

[elebioda]

Any thoughts on this bug?

[zbialik]

@ofirshtrull idk if it applies to you, but I got the helm values working by passing in the entire values.yaml as a parameter (multi-line string).

So for appset, you might have:

spec:
  template:
    spec:
      source:
        helm:
          values: '{{helmValues}}'

And the discovered config.yaml file via Git File Generator may have contents:

cluster_name: test-cluster
cluster_url: "https://8.8.8.8/"
cluster_env: "test"

helmValues: |
  nameOverride: kube-prometheus-stack
  ofir:
    i_work_to: yeap

[barthy1]

Unfortunately the mentioned workaround doesn't work for me with 2.4.10 . Any other suggestions?

[manuelchichi]

We are having this issue in our company. What we want to achieve is to have two different repositories in which one is private and the other is public to a specific client. Then we allow this client to freely edit values in the public one (to only allow changes on image tags). Finally, using the merge generator with two git generators we join both repositories and substitute the values of the tags.The workaround suggested here (multi line string) unfortunately allows our client to modify any value because the merge can't check the content inside the string. Also we can have different tags in our apps so we can't use parameters or such.

The merge generator creates the following yaml:

helmValues:
  nameOverride: kube-prometheus-stack
  ofir:
    i_work_to: yeap

In the application set we want to use it like this:

helm:
  values: |
    '{{helmValues}}'  #This Doesn't resolve

And the desired yaml would something similar to:

helm:
  values: |
    nameOverride: kube-prometheus-stack
    ofir:
      i_work_to: yeap

[joshuasimon-taulia]

i would like to iterate over a yaml list from a git file generator, but i can't get {{releases}}, {{releases[0]}}, {{releases.0}}, etc to render in the below example.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: abc-values-test
spec:
  generators:
  - git:
      repoURL: https://github.com/abc/def
      revision: HEAD
      files:
      - path: helmfiles/abc/helmfile.yaml
  template:
    metadata:
      name: '{{path.basename}}'
    spec:
      project: abc
      source:
        repoURL: https://github.com/abc/def
        targetRevision: HEAD
        path: '{{path}}'
        plugin:
          env:
          - name: HELM_TEMPLATE_OPTIONS
            value: --skip-tests
          - name: HELMFILE_GLOBAL_OPTIONS
            value: -l name={{releases[0]}} # or however the index is supposed to work
          - name: HELMFILE_USE_CONTEXT_NAMESPACE
            value: "true"
          name: helmfile
      destination:
        server: https://yolo.com
        namespace: '{{path.basename}}'
      ignoreDifferences:
      - group: admissionregistration.k8s.io
        jqPathExpressions:
        - .webhooks[].namespaceSelector.matchExpressions[] | select(.key == "control-plane")
        kind: MutatingWebhookConfiguration
        name: istio-sidecar-injector
      syncPolicy:
        syncOptions:
        - CreateNamespace=true

where releases in helmfile.yaml is

releases:
- chart: istio/base
  version: 1.12.2
  name: istio-base
  values:
  - jx-values.yaml
  - istio-base-values.yaml
- chart: istio/istiod
  version: 1.12.2
  name: istiod
  values:
  - jx-values.yaml
  - istiod-values.yaml

[joshuasimon-taulia]

it looks like the git file generator flattens the contents of maps/arrays so basically,

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: abc-values-test
spec:
  generators:
  - git:
      repoURL: https://github.com/abc/def
      revision: HEAD
      files:
      - path: helmfiles/abc/helmfile.yaml
  template:
    metadata:
      name: '{{path.basename}}-{{releases.0.name}}'

will work, but it seems like printing the entire '{{prometheus}}' map or looping over a list the way i want to is impossible

[Justin-DynamicD]

Adding voice to this as we are approaching a year later.

Is there a fix for this? Right now it seems like the appset wants to treat every map/list as a unique instance and therefore will only accept strings, which breaks pretty much every use case I would have for this. The biggest pain are the parameters that literally every app will need in order to differentiate deployments. How are we supposed to handle this?

Right now we are simply using a bash wrapper and running jq/yq and getting better results generating the applications directly, which is a shame, because Id love to toss those scripts.