rancher
Different Helm Chart version in one fleet.yaml with a local chart (relative path)
Is there an existing issue for this?
- [X] I have searched the existing issues
Current Behavior
I try to rollout a Helm chart on various clusters with different versions of the Helm chart. When my upgrade come from a local chart (relative path to a specific chart) it doesn't work and it's the same version installed on all clusters (which is defined in the main helm section of fleet.yaml).
Exemple fleet.yaml :
helm:
repo: https://romejoe.github.io/fleet-test/charts/
chart: version-test
version: "0.1.0"
releaseName: version-test
targetCustomizations:
- name: dev2
helm:
chart: ./version-3/
clusterSelector:
matchLabels:
env: dev
Normaly it was fixed by this issue #1011 but in this case it doesn't work. The problem is only with a local chart. This other use case works fine :
helm:
repo: https://charts.jetstack.io
chart: cert-manager
version: "1.9.0"
releaseName: cert-manager
targetCustomizations:
- name: dev3
helm:
repo: https://charts.jetstack.io
chart: cert-manager
version: "1.9.1"
clusterSelector:
matchLabels:
env: test
Regards
Expected Behavior
The new version is installed on the clusters which match the targetcustomization
Steps To Reproduce
Exemple fleet.yaml :
helm:
repo: https://romejoe.github.io/fleet-test/charts/
chart: version-test
version: "0.1.0"
releaseName: version-test
targetCustomizations:
- name: dev2
helm:
chart: ./version-3/
clusterSelector:
matchLabels:
env: dev
Environment
- Fleet Version: 0.7.1
- Cluster:
- Kubernetes Version:1.24.16 rke2
Logs
No response
Anything else?
No response
I believe I'm seeing this issue as well. I have a fleet.yaml that is constructed more-or-less like:
helm:
releaseName: my-release
chart: oci://registry1/org/my-chart
version: "1"
valuesFiles:
- values/values.yaml
targetCustomizations:
- name: airgapped
clusterSelector:
matchLabels:
fleet-airgapped: 'true'
helm:
chart: oci://airgapped-registry/org/my-chart
But on clusters that match the airgapped customization, the oci://registry1 repository continues to be used. I've confirmed that I can tweak other settings in the customization (such as adjusting the values) and it works -- it's just that the chart doesn't seem to be updated from a customization.
No further comments.
Please check with latest version and reopen if the problem still happens.
/reopen because this is still an issue in v0.11.2 of Fleet.
What ends up happening is that GitJob will first pull the Helm chart using the top-level helm.chart and helm.version attributes, and only after that, apply targetCustomizations when building the BundleDeployments.
This means that even though the constructed BundleDeployment will correctly show the updated chart value, GitJob didn't actually pull that chart -- it pulled the original one.
We don't intend to change this. However, we should document that targetCustomization cannot point to local folders.
It's not just a "local folder" problem -- the issue is that the chart: field of the bundle cannot be overridden at all by targetCustomizations. The toolchain pulls the Helm chart before targetCustomizations are processed.
I'm disappointed that this won't be addressed, as I believe we've noted relevant and important use cases for this feature.
It isn't a surprising or uncommon need for one environment to need to pull a different Helm chart than another environment. Whether it's the chart version/tag, or a different OCI registry storing the chart (e.g. due to firewalls in different deployment scenarios). Being able to use targetCustomizations to set this would be extremely useful and avoid a lot of complexity.
Today, we are forced to embed all of our Helm charts directly into Fleet bundles, specifically because a small chunk of our deployment environment is behind a firewall and thus can't use the same OCI registry as the rest of the environment. If we were able to use targetCustomizations to change that OCI registry endpoint, we could switch to a much cleaner and operations-friendly model of storing all of our Helm charts in an OCI registry.