[feature] HelmRelease load values from source controller artifact

[hikaru-shindo]

What problem would this feature solve

Currently we are deploying configuration for the OIDC authentication and authorization (basically a mapping of OIDC user ids to ClusterRoles) of our cluster admins and some read-only users via a helm chart to a fleet of clusters. Most clusters have similar settings (there are only a few different sets). As it stands now the helm-controller can only get the values (in this case the user id to cluster role mapping list) from either the release itself (values key) or a ConfigMap/Secret which has to be created first (as far as I understand it at least). We'd like to be able to share a set helm value files between different clusters via a separate artifact (which triggers a reconciliation of the user access on each affected cluster without us having to patch the values on each cluster ourselves).

What feature am I suggesting

For situations like this it would be nice if the HelmRelease resource/helm-controller could support pulling values from any artifact in the source controller. This could be realized with an API addition similar to this:

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
...
spec:
  valuesRef:
    - kind: OCIRepository
      name: my-config
      path: ./admins-only.yaml

This design would be similar to sourceRef and chartRef and therefore consistent with other places in flux2.

What issues might arise?

The value precedence (values vs valuesFrom vs valuesRef) would need to be discussed. What overrides what?

Additional Info

This might (or might not) be a special case as in #5167 at least it points into a similar direction.

ArgoCD would allow to solve this with a multi-source app approach maybe this helps in understanding the problem space a bit better:

apiVersion: argoproj.io/v1alpha1
kind: Application
...
spec:
  sources:
    - chart: my-chart
      repoURL: https://charts.example.com
      helm:
        valueFiles: 
        - $values/my-custom-values.yaml
    - repoURL: ...
      ref: values

[matheuscscp]

That OCIRepository could contain a kustomization generating a ConfigMap from that admins-only.yaml file and then you could deploy the ConfigMap with a Flux Kustomization, then you could pull in the values in HelmRelease with spec.valuesFrom

[hikaru-shindo]

Yeah that's an option but it seems kinda cumbersome. As I understand it the config map would need to be created before the helm release (dependsOn?) also this would potentially allow this kustomization to do more then providing the values (I'm thinking separation of concerns here - maybe in some other use case)

I still think this feature would be a more convenient and idiomatic way to do achieve that.

I'd also be willing to have a gander and try my luck creating a PR if the maintainers are okay with the idea in general.

[matheuscscp]

You can actually let the eventual consistency help you in this case, the ConfigMap and HelmRelease can be applied by the same Kustomization. If helm-controller happens to be faster than kustomize-controller and tries to reconcile the HelmRelease before kustomize-controller has time to apply the ConfigMap (which it is already about to do) and fails because of that, then the next reconciliation will most likely succeed, and it will kick in very soon because in this case we let the controller-runtime exp. backoff kick in rather than requeuing after a long interval. helm-controller is much slower than kustomize-controller so in most cases the first recon. attempt would succeed.

[stefanprodan]

The ConfigMap and the HelmRelease can be applied by the same Flux Kustomization and there is no problem with ordering, see the docs: https://fluxcd.io/flux/guides/helmreleases/#refer-to-values-in-configmaps-generated-with-kustomize

[hikaru-shindo]

@stefanprodan and @matheuscscp thanks for clearing up my misunderstanding here.

I still believe that this feels a bit strange if you just want to get some common organizational values to be used in your deployment I think.

It will probably be the workaround we will use for now but I still believe that a way to pull values in directly from any artifact that is configured in the source-controller would be a useful feature. It would simplify this setup and prevent the kustomizaton creating the ConfigMap as described by @matheuscscp to create any resources that are not expected.

I'm thinking something along the lines of a platform team applying some resources (e.g. role bindings for users as in my example above) but another team (e.g. a devx team) to provide the list of users and their organisational role. In this case I might want to be able to tell them to just generate a valid yaml configuration, put it in a bucket/oci/git repository and we just pull the platform team to just pull that configuration without them being able to influence anything but the the specified configuration.

I'd still be willing to do the PR if we can agree that this feature is/might be useful. If this is not determined to be a useful feature I'd rather not bother investing that time though.

[stefanprodan]

The user-case you described above can be accomplished with kustomize configMap generators, each team would push their own values file to the repo where the HelmRelease is defined.

All Flux APIs are designed with a single source, this change would require an RFC that must motivate why would we take on this complexity and what effects would it have on performance (dynamic Kubernetes watches, load on source-controller, etc).