helm-controller icon indicating copy to clipboard operation
helm-controller copied to clipboard

FEATURE: First-class support for secret decryption

Open davejbax opened this issue 3 months ago • 1 comments

Problem

Flux currently lacks first-class support for SOPS-encrypted secret decryption in Helm charts. If, for example, I wanted to load a Helm chart via a HelmRelease into my cluster and have it create some Kubernetes secrets from encrypted secrets stored in the Helm chart itself, the solutions available as I understand are:

  1. Use a Flux Kustomization which creates the HelmRelease and uses a secret generator to create the secrets, or
  2. Use an external secrets operator

Approach 1 ends up being slightly messy in a multi-cluster, multi-tenant scenario. Where Flux is used from a single centralised repo to pull application repos via HelmRelease resources and GitRepository sources, it becomes necessary to either:

  • Add the secret-generating Kustomization to the same repo storing the Helm chart code, which has the disadvantages of:
    • substantially increasing manifest boilerplate (e.g for each environment, kustomization.yaml, flux-ks.yaml, kustomizeconfig.yaml)
    • reducing local testability (since the Flux Kustomization is a CRD, it becomes cumbersome to use tools like Minikube without creating a local Flux installation)
  • Centralise all secret management in the centralised repo
    • e.g. apps/$app/overlays/$cluster/$env/{secret.enc,kustomization,kustomizeconfig,hr}.yaml
    • which is fairly untenable in a multi-cluster, multi-tenancy scenario in which we wish to decentralise as much workload-specific configuration as possible).

Approach 2 is feasible, but comes with its own disadvantages, which are covered fairly well in the Flux documentation on this topic.

Related issues

This issue has been raised in the past and discussed elsewhere online:

  • https://github.com/fluxcd/flux/issues/2804 (Flux 1)
  • https://github.com/fluxcd/flux2/discussions/3436
    • This uses Approach 1
  • https://www.reddit.com/r/kubernetes/comments/12g32kr/flux_helm_noob_here_how_do_i_pass_secret_values/

Proposal

I'd like to propose a cleaner solution for handling this case:

  1. Add an additional field to HelmRelease, named something such as valuesFilesEncrypted
  2. Decrypt these values files with SOPS when building the Helm chart
    • E.g. Around the following locations:
      • https://github.com/fluxcd/source-controller/blob/0c81d04c8939903d2861a70c62571208afc9c303/internal/helm/chart/builder_local.go#L144
      • https://github.com/fluxcd/source-controller/blob/0c81d04c8939903d2861a70c62571208afc9c303/internal/helm/chart/builder_remote.go#L106

The ostensible benefits of this approach would be:

  • Removing the need to mix Flux Kustomizations and HelmReleases within a single repo
  • Secret management with SOPS in Helm charts becomes much more accessible
  • Simplicity
  • Backwards compatibility: the field would be optional, and no change in behaviour would occur if it is unset (which would be the default)

I'm happy to open a PR to implement the above, if the maintainers believe it is an acceptable idea to implement.

davejbax avatar Mar 31 '24 14:03 davejbax