helmfile
helmfile copied to clipboard
Is this possible to clear a property previously defined ?
let's say i have something along those lines :
values:
- nodeSelector:
nodeType: infra
- nodeSelector: null
in the end, nodeSelector
is not reset to null
.
is this possible to achieve something like this ?
@eddycharly Hey! Unfortunately no, it isn't possible and I have no idea how this can be done elegantly.
But what would be your use-case? Perhaps I can suggest alternatives.
@mumoshu Hi, my use case is this one : I have a base layer where I describe my platform infrastructure (more or less a combination of feature flags and helm values), and a specific layer for my environments to enrich/customise what was declared in the base layer on a per environment basis.
My helmfile.yaml is a template that generates all that is necessary for helmfile based on the abstract environment definition.
In the base layer I put nodeSelector statements to drive the placement of the workloads for the releases. But now I am creating a local environment that run on a single cluster, I want to clear nodeSelector statements.
I’ll write an example in a few minutes it will be easier to understand ;)
@mumoshu let's say my environments/_base.yaml
is like this :
core:
clusterAutoscaler:
enabled: true
nodeSelector:
kops.k8s.io/instancegroup: infra
dashboard:
enabled: true
nodeSelector:
kops.k8s.io/instancegroup: infra
externalDns:
enabled: true
nodeSelector:
kops.k8s.io/instancegroup: infra
heapster:
enabled: true
nodeSelector:
kops.k8s.io/instancegroup: infra
metricsServer:
enabled: true
nodeSelector:
kops.k8s.io/instancegroup: infra
nodeProblemDetector:
enabled: true
nodeSelector:
kops.k8s.io/instancegroup: infra
then my "local" environment environments/microk8s.yaml
:
clusterName: microk8s
kubeContext: microk8s
core:
clusterAutoscaler:
enabled: false
dashboard:
enabled: true
nodeSelector: null
externalDns:
enabled: false
heapster:
enabled: false
metricsServer:
enabled: false
nodeProblemDetector:
enabled: false
and finally my helmfile.yaml
:
environments:
microk8s:
values:
- environments/_base.yaml
- environments/microk8s.yaml
---
repositories:
- name: stable
url: https://kubernetes-charts.storage.googleapis.com
helmDefaults:
kubeContext: {{ .Values.kubeContext }}
verify: false
wait: true
recreatePods: true
force: true
releases:
# CORE
{{ if .Values.core.dashboard.enabled }}
- {{ tpl (readFile "templates/core/dashboard.yaml") . | nindent 4 }}
{{ end }}
{{ if .Values.core.heapster.enabled }}
- {{ tpl (readFile "templates/core/heapster.yaml") . | nindent 4 }}
{{ end }}
{{ if .Values.core.externalDns.enabled }}
- {{ tpl (readFile "templates/core/external-dns.yaml") . | nindent 4 }}
{{ end }}
{{ if .Values.core.metricsServer.enabled }}
- {{ tpl (readFile "templates/core/metrics-server.yaml") . | nindent 4 }}
{{ end }}
{{ if .Values.core.nodeProblemDetector.enabled }}
- {{ tpl (readFile "templates/core/node-problem-detector.yaml") . | nindent 4 }}
{{ end }}
{{ if .Values.core.clusterAutoscaler.enabled }}
- {{ tpl (readFile "templates/core/cluster-autoscaler.yaml") . | nindent 4 }}
{{ end }}
of course there are a lot more sections than core
, i have sections for logging, monitoring, infrastructure, backups, ingress...
what i was trying to do was to define the default nodeSelectors
in the environments/_base.yaml
and eventually override them in the environment specific file if needed.
@eddycharly Thanks for clarifying!
I believe what you are trying to achieve is out of the scope of YAML or configs expressible with Helmfile's API.
But I believe you can still use go templates and sprig functionsou to achieve anything beyond YAML/Helmfile's API.
Try sprig's unset to unset any key without a value.
{{ $overridesFile := printf "environments/%s.yaml" .Environment.Name }}
{{ $base := readFile "environments/_base.yaml" | fromYaml }}
{{ $overrides := readFile $overridesFile | fromYaml }}
{{ $merged := mergeOverwrite (dict) $base $override }}
{{ $keys = keys $merged }}
{{ _, $key := range $keys }}
{{ $val := (pluck $key $merged | first) }}
{{ if not $val }}
{{ unset $merged $key }}
{{ end }}
{{ end }}
environments:
{{ .Environment.Name }}:
values:
-
{{ toYaml $merged | indent 6 }}
---
repositories:
- name: stable
url: https://kubernetes-charts.storage.googleapis.com
...
@mumoshu for what it’s worth, helm handles null values as a special case, it removes a key instead of merging it https://github.com/helm/helm/blob/master/docs/chart_template_guide/values_files.md#deleting-a-default-key
@eddycharly Wow, that's good to know!
I've took another look, and it turned out https://github.com/imdario/mergo/issues/115 needs to be resolved first. Helmfile uses mergo for merging values entries and mergo doesn't support overrinding will nil(yaml null
)s yet.
Also - does - nodeSelector: []
instead of - nodeSelector: null
work for this specific case? I guess you'd have something like {{if .Values.nodeSelector }}...{{end}}
within your chart template(not helmfile template) and the if condition turns false when nodeSelector
is an empty array, which means you can use empty arrays as alternatives to nulls in this case.
@mumoshu yes, using an empty array in the nodeSelector does the trick. I am not a big fan of this solution though.
For now I ended up with nodeSelector config at the environment level, not in the base definition. It’s a bit more typing but not a big deal.
Anyway I feel like I’m abusing the helmfile templating feature, i think it goes too far into abstracting helm details, and relying on helmfile templating to generate a valid helmfile based on my own definition is somewhat unadapted.
I’m thinking about moving this process in a separate job, using something like gomplate to produce the desired helmfile first, then apply the produced configuration with helmfile.
It should be easier to debug also.
@eddycharly Thanks for the confirmation and sharing your insights!
relying on helmfile templating to generate a valid helmfile based on my own definition is somewhat unadapted. I’m thinking about moving this process in a separate job, using something like gomplate to produce the desired helmfile first, then apply the produced configuration with helmfile.
I fully agree. YAML + Go template is workable, easy to start, but not the best solution especially when you have advanced use-cases.
I'm considering various options like Jsonnet(#814) and CUE(#869) for that. In the context of Helmfile, I started liking CUE. If you're interested, see examples shown in the issue!
@eddycharly FWIW, helmfile build
helps debugging advanced configs containing many go template expressions, by printing the rendered YAML.
Still, I'm thinking using Jsonnet or CUE is better in terms of expressiveness and readability.
@mumoshu hi, in my case it seem to be override with {} does not work as well, I have following value lookup hierarchy:
values:
- "../values/common.yaml.gotmpl"
- "../values/{{ requiredEnv "ENVIRONMENT" }}/common.yaml.gotmpl"
at default level in values/common.yaml.gotmpl I have default values for resources defined:
resources:
requests:
cpu: 200m
memory: 600Mi
limits:
cpu: 400m
memory: 800Mi
and then under specific environement layer I'm trying to override resources with empty value {}
: values/production/common.yaml.gotmpl
resources: {}
Unfortunately resulting template for the production will not have resources set to {} but will contain values from common.yaml defined at default level: values/common.yaml.gotmpl
I assume this is result of this issue : https://github.com/roboll/helmfile/issues/866#issuecomment-533930988 or maybe it's something entirely different? Thanks
Any news about this issue?
I'm looking forward to a feature like Helm does on Deleting a default key.
In my case, the sub chart I was using had a default CPU limit set in its values.yaml. Following the 4th workaround mentioned here worked for me, as given below:
helmfile:
releases:
- name: refinery
namespace: tracing
chart: PARENT_CHART
version: 1.2.4
values:
- values/refinery.yaml
- refinery:
resources:
limits:
cpu: null
values/refinery.yaml
refinery:
resources:
requests:
cpu: 3000m
memory: 6Gi
limits:
memory: 6Gi
PARENT_CHART/values.yaml
refinery:
resources:
limits:
cpu: null
memory: 2Gi
requests:
cpu: 2000m
memory: 2Gi
Any updates on this topic? I would be great to always be aligned with the helm feature set. I have a similar problem and have to move now back to helm as there seems to be no way to remove default values right now. https://github.com/imdario/mergo/issues/115 is merged btw, any idea when this could be released in helmfile? @mumoshu mumoshu