helmGlobals is not used when HelmChartInflationGenerator is used in generators field

[LarsStegman]

What happened?

I am having trouble getting my deployments of a Helm chart to multiple sites/deployments working.

The setup

/
├─ charts/
│  ├─ my-app/
├─ my-app/
│  ├─ helm/
│  │  ├─ helm.yaml
│  │  ├─ kustomization.yaml
│  ├─ overlays/
│  │  ├─ site1/
│  │  │  ├─ common/
│  │  │  │  ├─ kustomization.yaml
│  │  │  ├─ deployments/
│  │  │  │  ├─ deployment1/
│  │  │  │  │  ├─ kustomization.yaml
│  │  │  │  ├─ kustomization.yaml

I have this structure because I need to deploy my app to multiple sites. Some of the values need to be the same across all sites, like the application version, for example. Some of the values need to be the same across 1 site, like some credentials/domain names. Some of the values are specific per deployment, like the name.

This is the contents of the /my-app/helm folder. It is used as the base
config for all deployments.

# /my-app/helm/helm.yaml
apiVersion: builtin
kind: HelmChartInflationGenerator
metadata:
  name: my-app
name: my-app

# /my-app/helm/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - helm.yaml

Then, per site I kustomize the HelmChartInflationGenerator from /my-app/helm/helm.yaml to provide some valuesInline.

# /my-app/overlays/site1/common/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../../helm

patches:
  - patch: |-
      apiVersion: builtin
      kind: HelmChartInflationGenerator
      metadata:
        name: my-app
      valuesInline:
        k8sSecret:
          secretName: my-secret
        outputs:
          - url: "http://database.site1.company.global"
            tokenSecretKey: write1
            db: "db1"

Then in /my-app/overlays/site1/deployments/deployment1/kustomization.yaml I kustomize the same HelmChartInflationGenerator another time to add additional valuesInline values.

# /my-app/overlays/site1/deployments/deployment1/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../common-config

patches:
  - patch: |-
      apiVersion: builtin
      kind: HelmChartInflationGenerator
      metadata:
        name: my-app
      releaseName: deployment1-my-app
      valuesInline:
        requireActiveAccessToSensors: true
        socketListeners:
          - protocol: UDP
            port: 2000
        appConfig: |
          [config]
            key1 = "value1"

Finally, I collect all the different deployments from the site in /my-app/overlays/site1/deployments/kustomization.yaml.

# /my-app/overlays/site1/deployments/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
helmGlobals:
  chartHome: ../../../charts

generators:
  - ./deployment1

The problem

When I try to build this kustomization I get the following:

> cd my-app/overlays/site1/deployments
> kustomize build . --enable-helm --load-restrictor LoadRestrictionsNone
Error: no repo specified for pull, no chart found at ''

It looks like helmGlobals.chartHome is not passed to the HelmChartInflationGenerator at all. Even if I change it to foo, I still get the same error. I would expect it to say no chart found at 'foo' then.

Am I doing something wrong, or is this a bug?

What did you expect to happen?

The Helm chart is inflated multiple times, once per sites per deployment.

How can we reproduce it (as minimally and precisely as possible)?

Files

# /my-app/helm/helm.yaml
apiVersion: builtin
kind: HelmChartInflationGenerator
metadata:
  name: my-app
name: my-app

# /my-app/helm/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - helm.yaml

# /my-app/overlays/site1/common/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../../helm

patches:
  - patch: |-
      apiVersion: builtin
      kind: HelmChartInflationGenerator
      metadata:
        name: my-app
      valuesInline:
        k8sSecret:
          secretName: my-secret
        outputs:
          - url: "http://database.site1.company.global"
            tokenSecretKey: write1
            db: "db1"

# /my-app/overlays/site1/deployments/deployment1/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../common-config

patches:
  - patch: |-
      apiVersion: builtin
      kind: HelmChartInflationGenerator
      metadata:
        name: my-app
      releaseName: deployment1-my-app
      valuesInline:
        requireActiveAccessToSensors: true
        socketListeners:
          - protocol: UDP
            port: 2000
        appConfig: |
          [config]
            key1 = "value1"

# /my-app/overlays/site1/deployments/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
helmGlobals:
  chartHome: ../../../charts

generators:
  - ./deployment1

Command

> cd my-app/overlays/site1/deployments
> kustomize build . --enable-helm --load-restrictor LoadRestrictionsNone
Error: no repo specified for pull, no chart found at ''

Expected output

Inflated Helm Chart per site per deployment

Actual output

An error saying the Helm chart cannot be found

Kustomize version

5.3.0

Operating system

Linux

[k8s-ci-robot]

This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[busilina]

I had a similar problem. After checking the source code I discovered that (in my case) the folder name where my chart resides was not correct.

Have a look at this: https://github.com/kubernetes-sigs/kustomize/blob/c1de0301f5e71ffbecf77f5bc8687e7693878eb5/plugin/builtin/helmchartinflationgenerator/HelmChartInflationGenerator.go#L343

[LarsStegman]

I don't think that is actually the problem in my case, because I think I tested that, but it has been a while. I will report back in few weeks.