flux2
flux2 copied to clipboard
HelmRelease should support SOPS encryption directly
Describe the bug
Even if we have workaround described here -> https://github.com/fluxcd/flux2/discussions/1537
Users of SOPS are used to not have to change file format, or move part of configurations to be able to encrypt the few things that need encryption.
It should be straightforward to support that HelmRelease be decrypted on the fly if it's encrypted.
Steps to reproduce
When assuming this is already implemented, the error is a little vague, it's saying: dry-run failed, error: failed to create typed patch object (...; helm.toolkit.fluxcd.io/v2beta1, Kind=HelmRelease) .sops: field not declared in schema
Expected behavior
SOPS should just work, without having user to be confused by error message, had to Google, then implement workarounds, splitting a consistent and coherent YAML to many.
Screenshots and recordings
No response
OS / Distro
Linux
Flux version
flux: v0.41.2
Flux check
► checking prerequisites ✗ flux 0.41.2 <2.0.1 (new version is available, please upgrade) ✔ Kubernetes 1.27.1 >=1.20.6-0 ► checking controllers ✔ helm-controller: deployment ready ► ghcr.io/fluxcd/helm-controller:v0.31.1 ✔ kustomize-controller: deployment ready ► ghcr.io/fluxcd/kustomize-controller:v0.35.0 ✔ notification-controller: deployment ready ► ghcr.io/fluxcd/notification-controller:v0.33.0 ✔ source-controller: deployment ready ► ghcr.io/fluxcd/source-controller:v0.36.0 ► checking crds ✔ alerts.notification.toolkit.fluxcd.io/v1beta2 ✔ buckets.source.toolkit.fluxcd.io/v1beta2 ✔ gitrepositories.source.toolkit.fluxcd.io/v1beta2 ✔ helmcharts.source.toolkit.fluxcd.io/v1beta2 ✔ helmreleases.helm.toolkit.fluxcd.io/v2beta1 ✔ helmrepositories.source.toolkit.fluxcd.io/v1beta2 ✔ kustomizations.kustomize.toolkit.fluxcd.io/v1beta2 ✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2 ✔ providers.notification.toolkit.fluxcd.io/v1beta2 ✔ receivers.notification.toolkit.fluxcd.io/v1beta2 ✔ all checks passed
Git provider
No response
Container Registry provider
No response
Additional context
No response
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
I think there's some alignment issues with proliferating the decryption capability beyond where it currently applies, barring any technical implementation issues that might arise: Kustomize controller is the only thing that supports decryption, and it happens at kustomize build / kubectl apply time. I see you found a thread where that was discussed, and I understand you wish to see HelmReleases encrypted directly if the user wills it when they are in the git repository...
The decryption capability is only supported for secrets, because the secret is expected to contain secrets. A HelmRelease definition is not expected to contain secrets in it. There are concerns that go beyond the level of mere semantics, if you think it doesn't matter if you keep a secret somewhere other than a secret; etcd data is generally not encrypted, but you can enable encryption for secrets. But if you store secret data somewhere other than in a secret, in that case it would be stored unencrypted at rest on the cluster's etcd data store. This would be, to say the very least, sub-optimal.
I don't think this suggestion is likely to be implemented without further justification (or at all.) Flux follows this standard to promote alignment; if your secrets are encrypted in the git repository, then by all rights they should probably be encrypted at rest in the cluster too, but that's something beyond Flux's control. Flux's restriction where you found it on which types of resources can be encrypted and decrypted, is intentional.
The error messaging when you have stepped outside of this is maybe not so helpful, which is why I don't close this, maybe we can do better at messaging the failure reason (or maybe we do close it with a docs update, and also this issue can serve as an explainer for others that find it later.)
We can add some explanation about what types of resource are allowed here, and anywhere else you think it would help?
https://fluxcd.io/flux/flux-e2e/#secret-decryption-via-sops
@kingdonb said: Kustomize controller is the only thing that supports decryption
I use Helm to deploy everything, even secret, and as you can guess, the Helm values is where I found all my encrypted values. It's very natural to expect them there. It's the main benefit of Mozilla SOPS compared to convoluted solution like HashiCorp Vault, you can have all the config including one or two line encrypted in the same file, in the same normal/natural order/location.
I used enough Helm charts to know that many values, that should not be in cleartext, are not expected to be in a secret by the Helm chart.
For example, from mysql chart, the first random Helm chart I selected to make this point:
- https://github.com/helm/charts/blob/master/stable/mysql/values.yaml#L150
- https://github.com/helm/charts/blob/master/stable/mysql/values.yaml#L219
@kingdonb said: if you think it doesn't matter if you keep a secret somewhere other than a secret
The Helm chart manifest/history are not persisted in a configmap, but in a secret. So I still don't see the problem.
Also, I'm not aware of any serious implementation where the etcd will be persisted on a drive that is not encrypted. The feature of having part of etcd with different encryption seems to be rarely used.
Can we expect this to be done in near future?
Can we expect this to be done in near future?
No. Adding SOPS to helm-controller is a huge dev effort. IMO the Flux core team should focus this year on the milestones we have defined in the Flux roadmap https://fluxcd.io/roadmap/