helm-controller icon indicating copy to clipboard operation
helm-controller copied to clipboard

feat: More flexibility when applying charts to remote clusters

Open mclarke47 opened this issue 7 months ago • 1 comments

I am experimenting with flux but am concerned with the flexibility of applying to remote clusters.

I have a multi-tenant admin configuration cluster where my Flux objects live and a multi-tenant workload cluster where workloads actually run. I am currently deploying via a HelmRelease per remote workload cluster, and I create a kubeconfig secret to contain the credentials to apply to the remote cluster.

However, this isn't very easy since HelmReleases cant seem to reference secrets in another namespace. Therefore, I need to create multiple secrets for each helmrelease namespace which is pretty complicated. It would be great if the helm controller could:

  • Use Workload Identity like some of the other flux components for pulling OCI repositories (the Providerfield
  • Specify a central secret used to deploy to remote clusters

With guidance, I would be willing to submit a PR to add this feature.

mclarke47 avatar Nov 29 '23 16:11 mclarke47

I suggest using some controller that knows how to copy secrets between namespaces, for example: https://kyverno.io/policies/other/s-z/sync-secrets/sync-secrets/

When Kubernetes will allow a Deployment to refer to Secrets from another namespace, then we'll consider allowing it in Flux too. I doubt this will ever happen upstream as it break namespace isolation and RBAC.

stefanprodan avatar Dec 01 '23 13:12 stefanprodan