Handle vals failures if a reference is unavailable

[andrewnazarov]

Say I have the following reference:

privateKey: ref+gcpsecrets://blablabla

By default, if a referenced value is not presented helmfile template or helmfile apply will fail. However sometimes it's ok, we know that we don't have these resources yet and we can live without them. Could we have something similar to missingFileHandler(for go-getters) or to disableValidation(for crds) ?

[andrewnazarov]

This looks almost the same at first glance: https://github.com/variantdev/vals/issues/52

[mumoshu]

@andrewnazarov Hey! I was thinking about this recently and have come up with a few options:

1. Helmfile: A release-level option like missingFileHandler, e.g. missingValsValueHandler?
2. Helmfile: Add a second parameter to {{ fetchSecretValue "ref+..." }} function so that it does not fail but instead returns the value from the second arg as the default value
3. Vals: Add additional common option to make a value optional. Like ?optional so that ref+gcpsecrets://blablabla?optional can result in an empty string on missing ref

The third has some potential to resolve https://github.com/variantdev/vals/issues/52 but I'm not yet confident if that's the right way to go.

If you need https://github.com/variantdev/vals/issues/52 for use from Helmfile only, the second seems to be more consistent as we already have {{ get }} function that takes the second parameter and works similarly.

WDYT?

[andrewnazarov]

@mumoshu Hi:)

Yes, we only use vals as a part of helmfile, therefore I'd share your concern. And the second option looks better for me too. Haven't used fetchSecretValue function before though:)

When I initially mentioned an equivalent of disableValidation I thought that there might be use cases when something (a secret or information that can be stored as a secret) is created alongside a release that has a ref+ link either a part of this release or as a separate release. And since on a first run this referenced resource doesn't exist we can't proceed further. But then I realised that it wouldn't work either, because a manifest is rendered prior to helm diff. In the end, I believe it can be solved via reconsidering the approach. Not sure how helmfile can help here, probably it shouldn't.

[iAnomaly]

@mumoshu +1 from me for option 2 at least for resolving this issue in Helmfile.

Not sure how best to resolve it in Vals. Maybe 3 would be work if it could return nil instead of empty string? I would be afraid of cases where ref+gcpsecrets://blablabla?optional key exists but whose value is already empty string.

[mumoshu]

@iAnomaly Good point! Yeah, I'd try 3 as the first option, with the caveat in my mind.

[iAnomaly]

Do you have time to take this on soon @mumoshu or are you looking for a PR from the community?

[mumoshu]

@iAnomaly I tend to await PRs from the community with associated issues with enough discussions(like this one) so that we can better co-maintain this project :) Thanks for confirming!

[poligraph]

Hi @mumoshu,

I faced the same issue today while do some condition in helmfile

so before, I used sops encrypted file which was defined in secrets section

it worked if secret exists or not:

{{ if .Values | get "secret" false }}
   service:
    mysecret: {{ .Values.secret }}
 {{ end }}

I have switched to Google Secret Manager and moved secrets to values and there is bit different syntax. If secret exist - it works:

 {{ if fetchSecretValue Values.secret }}
   service:
    mysecret: {{ fetchSecretValue .Values.secret }}
 {{ end }}

but if not exists -- it will fail.

I would like that function fetchSecretValue behaviour should be the same as get and following code should work:

 {{ if fetchSecretValue Values.secret false}}
   service:
    mysecret: {{ fetchSecretValue .Values.secret }}
 {{ end }}

Any chance to implement it?