helmfile

helmfile copied to clipboard

what

• Add option to process helmfile.yaml, but not execute helm

why

• This would be helmfile for development

use-case

We're working on a master helmfile, kind of like a "distribution" of charts. Part of this involves a lot of parameterization. It would make it easier to safely iterate using a --dry-run flag as is common with many command line tools.

@osterman Hi! Thanks for trying helmfile.

Would you mind elaborating a bit more about your expected output from helmfile?

I guess it suffices your expectation if helmfile diff is extended to show diff even in case there's no installed releases yet?(Btw it requires an enhancement in helm-diff side)

Or maybe you'd want something like ver=0.4.0 bash -c 'helm fetch stable/mysql --version $ver && helm template mysql-$ver.tgz' for all the releases defined in helmfile.yml?

In this particular use-case, I don't even have access to kubernetes or helm. I just want to validate the helmfile.yaml template results is valid yaml and that the --set arguments are correct.

@osterman Thanks for the clarification!

Just to be certain, would you mind running ver=0.4.0 bash -c 'helm fetch stable/mysql --version $ver && helm template mysql-$ver.tgz'?

The output contains the rendered kubernetes manifests which should reflect everything you provided - chart and values. So I guess you can use that for the validation? If it makes sense, I'm eager to implement --dry-run which outputs something like that for every release.

Just to be extra clear, with the upcoming 0.13.0 release, the entire helmfile.yaml is a template. We're using the current master. No mechanism exists to validate the output from the sprig template and thus the execution of helm upgade..... from the generated yaml.

I want to know what helmfile will execute before helmfile executes anything.

This is kind of like rsync --dry-run. I am familiar with helm template as I use that when I iterate on my charts. Right now, I'm developing a very advanced helmfile.yaml but I lack the ability to validate the templatized yaml.

Currently, helmfile will output: (and a subsequent call to helm)

exec: helm upgrade --install charts stable/chartmuseum .............

Instead, I want to see: (with no execution of helm)

[dry-run] exec: helm upgrade --install charts stable/chartmuseum .............

For example, I've tried using sprigcli to render the template, but it doesn't implement coalesce or uses an old version.

Here's an example of the complex helmfile.yaml: https://github.com/cloudposse/geodesic/pull/124/files

Thanks! Got it. Yeah, such feature would be useful 👍

Would it make sense to add a helmfile lint similar to helm lint. That could do validating and maybe other checking?

@sstarcher I think so. Actually, we have the exact discussion in #58 😉

Yes, that sounds like a great idea. The lint would satisfy the helmfile validation. That said, I still want to know what the command looks like before executing it as the helmfile syntax is a big divergence from a standard values.yaml.

@osterman Agreed I was thinking we needed a command to output a generated helmfile. We could either treat this as a part of lint. helmfile lint --generate or as a separate command helmfile generate

I guess the equivalent would be helms helm install --dry-run --debug. Possibly a helmfile lint --debug to lint and output the generated helmfile?

@sstarcher @osterman Just to be extra clear, I was thinking that --dry-run and lint to be two different things.

So, I'd expect:

• 👍 helmfile sync --dry-run to do everything but running helm upgrade --install - it renders helmfile and print the commands without actually running them. I believe this is what @osterman like via this feature request.
• 👍 helmfile lint to render helmfile and run helm lint. It doesn't print the rendered helmfile normally, but it does print helmfile if the lint failed. Also, we probably won't add helmfile lint --dry-run as there seems like no motivation to not actually running helm lint.
• 🤔 helmfile lint --debug would print the rendered helmfile even if there was no lint error. But given the vanilla helmfile lint prints the rendered helmfile on a lint error, I think we have no specific need for this feature?
• 🤔 helmfile template(after helm template) to output a generated helmfile. But I'm not sure about an exact use-case for this feature too, because the vanilla helmfile lint would print the rendered helmfile when necessary.

so helmfile sync --dry-run would not do helm upgrade --install --dry-run

@sstarcher Thanks! That's exactly my first thought. But given helm upgrade --install -dry-run doesn't bloat up the output much:

$ helm upgrade --install --dry-run foo stable/mysql
Release "foo" does not exist. Installing it now.
NAME:   foo

I started to think I can alter my proposal to:

• 👍 helmfile sync --dry-run to do everything but actually running helm upgrade --install - it renders helmfile and print the commands, run helm upgrade --install -dry-run without actually ~running~installing them. I believe this is what @osterman like via this feature request.
• 👍 helmfile sync --dry-run --debug to do everything but actually running helm upgrade --install - it renders helmfile and print the commands, run helm upgrade --install -dry-run --debug to print the whole k8s manifests rendered by helm, without actually ~running~installing them.
• 👍 helmfile lint to render helmfile and run helm lint. It doesn't print the rendered helmfile normally, but it does print helmfile if the lint failed. Also, we probably won't add helmfile lint --dry-run as there seems like no motivation to not actually running helm lint.

How about this?

I think that sounds good.

For clarification, are you saying that helmfile sync --dry-run would also call helm upgrade --install --dry-run?

Basically, in my use-case, I don't even have helm installed and just want to see the commands, so I would not want it to call helm upgrade --install --dry-run. I think executing the helm upgrade --install --dry-run could be accomplished instead by running:

helmfile sync --args '--dry-run'

Thus my conclusion is helmfile sync --dry-run should never call exec.

@osterman

For clarification, are you saying that helmfile sync --dry-run would also call helm upgrade --install --dry-run?

Exactly!

Basically, in my use-case, I don't even have helm installed and just want to see the commands

Yes, I think I understand the value of that. Good point!

However, to me it seems like that we can't achieve that perfectly by just suppressing helm upgrade install --dry-run calls.

You'd occasionally need to encrypt/decrypt secrets referenced via secrets: section, which results in helm secret dec calls.

So, even we stopped calling helm upgrade install --dry-run in helmfile sync --dry-run, helmfile stills needs helm binary to call helm-secrets.

I can't list every edge-case like that right now but I do worry about them. Is the ability to dry-run helmfile without a helm binary a hard requirement for you?

Thanks!

you'd occasionally need to encrypt/decrypt secrets referenced via secrets: section, which results in helm secret dec calls.

Aha, yes, we use chamber instead, which is why this didn't occur to me.

So, even we stopped calling helm upgrade install --dry-run in helmfile sync --dry-run, helmfile stills needs helm binary to call helm-secrets.

Makes sense. I don't have a workaround for that.

Is the ability to dry-run helmfile without a helm binary a hard requirement for you?

No, it's just a preference. =)

It seems reasonable for people to want to dry-run the helmfile without having a helm server.

@sstarcher In my understanding, helm upgrade --install --dry-run ddepends on tiller(helm server) but helm lint is not.

So, how about serving people who want to dry-run helmfile without tiller with helmfile lint?

@mumoshu I want to see the rendered helmfile so I can validate that template conditionals are working as expected. With a complicated helmfile that generates multiple charts, it can be tough to trace back errors to the helmfile. And by errors I do not mean syntax or lint errors, I mean bugs where the helmfile is not doing what I want it to do.

any progress on this, would love to have --dry-run to just see the generated yaml

I need to settle on a spec to make progress 😃

@bitsofinfo What do you want for the "generated yaml"? Is it a dump of rendered helmfile.yaml, or basically a collection of helm template outputs, or anything else?

@Nuru @bitsofinfo Can I assume that what you need is something like helmfile --log-level=debug sync --dry-run, that dumps the rendered helmfile.yaml(not k8s manifests generated by helm templates), without actually applying them?

In my particular use case, I just wanted the same out put as helm install --dry-run --debug ... which yes, the generated templates w/out running them. Maybe new args for helmfile like --helm-dry-run and --helm-debug which just pass through? (as to not collide w/ your own args)

@bitsofinfo Thx! Perhaps helmfile template helps in that case?

@mumoshu As a general rule, any tool that does multiple processing passes should have a way to stop at the end of any given pass and output what will be the input to the next pass. That includes outputting the output of the last input processing phase before acting on it. It might help you both architecturally and from UI and logical modeling perspective to think of it that way.

@Nuru Thanks for sharing your insight!

Would you mind also sharing an example of a helmfile command output that conforms to the rule?

@mumoshu I'm not sure what you are asking for. I think for an example you can look at Gnu C++, which will output the result of the preprocessor, (back in the days when it was translated into C, the output of the translation to C), the translation into machine language, and the output as object files before being fed into the linker. So for helmfile I'd at least expect to be able to see the output it plans to send to helm without it actually sending it to helm.

@Nuru Thanks! So helmfile sync --dry-run may only print:

skipped: helmfile upgrade --install ....
skipped: helmfile upgrade --install ....
skipped: helmfile upgrade --install ....
skipped: helmfile upgrade --install ....

helmfile destroy --dry-run may print:

helmfile list ^yourrelease1$
skipped: helm delete --purge yourrelease1
helmfile list ^yourrelease2$
helmfile list ^yourrelease3$

Is that what you actually want? I'm just not sure what's you're exactly asking for!

In other words, I'm not asking for theory, but wanting to discuss about UI design and implementation idea for --dry-run.