Optionally enable external plugins

[polivbr]

External plugins are a very useful and power feature of kustomize, but due to their current alpha status, are not not enabled by default.

This commit allows an optional pluginHome path to be specified, relative to the source root, which will enable the plugin functionality.

[polivbr]

Addresses issue #120

[stefanprodan]

Hmm so people are expected to place Linux binaries in their repo then execute them inside the controller container. I wander if this is a miss use of Git, let's discuss this at the next Flux meeting.

[polivbr]

There's nothing saying that the executables have to be linux binaries. They could be shell scripts, which to me is no different than having CI build scripts in your git repo. Even if someone wanted to use binaries, that should be the decision of the repo owner and shouldn't be dictated by the tooling.

[zhech2]

My team is interested in this change. Is there anything we can do to help?

[mhaley-tignis]

Any way we can push this forward? Our team is using the --enable-helm option with kustomize when running locally. I would like to use the same manifests in our cloud clusters managed by fluxv2 but currently I get a reconciliation error:

{"level":"error","ts":"2021-09-14T15:15:13.101Z","logger":"controller.kustomization","msg":"Reconciliation failed after 42.506436ms, next try in 10m0s","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"aipc","namespace":"flux-system","revision":"main/d4e9662ac29fb45bb9a61bba662e8ba815b60a4f","error":"kustomize build failed: accumulating resources: 2 errors occurred:\n\t* accumulateFile error: \"accumulating resources from '../../components/cert-manager-installation': read /tmp/aipc237354456/infrastructure/components/cert-manager-installation: is a directory\"\n\t* accumulateDirector error: \"couldn't make target for path '/tmp/aipc237354456/infrastructure/components/cert-manager-installation': json: unknown field \\\"helmCharts\\\"\"\n\n"}

Being able to set the plugin config passed into kustomize build would solve our problem. There is an enabled plugin config already (see here). Can we create an option that allows the controller to use that configuration instead of the default?

I don't see this changing security threat model at all since the helm binary being executed has to be baked into the container. As long as it is trusted, and works as intended, things should be fine.

[khrisrichardson]

@mhaley-tignis This PR looks to be more specific to running exec plugins. A patch I reference in https://github.com/fluxcd/kustomize-controller/issues/323#issuecomment-913767721 enables both the KRM config function plugins and the Helm generator. I agree that those could be handled by kustomize-controller command line flags, with the caveat that if someone wishes to use the KRM config functions, they'll need to provide a gotk-patches.yaml file to mount /var/run/docker.sock, which would be a security concern, albeit not germane to your request to enable the Helm generator. The other caveat that is germane to your request is that the kustomize-controller container image doesn't ship with the Helm binary ... another reason I went the custom image route.

[mhaley-tignis]

You are right. I think we are just going to switch to running helm template ourselves on the few charts we rely on, checking the resulting manifest in, then just using kustomize like we do for everything else, and patching the manifest if/when needed. When the external chart is updated, our upgrade path should be as simple as helm template again (minus all the other complications of live updates :) ).

[akokshar]

Are there any plans to merge this? Not being able to use self-written exec plugins is a blocking obstacle for us to adopt fluxcd :/

[stefanprodan]

I will close this PR when we get OCI artifacts support into Flux. Using OCIRepository instead of GitRepository will enable a workflow where you run kustomize build and flux push in CI, then you reconcile the final manifests with a Flux Kustomization that uses the OCI artifact as source.

The workflow goes like this:

• you commit a change to Git
• CI runs kustomize build --enable-alpha-plugins --enable-exec --enable-helm . > ./app-deploy/final.yaml
• CI runs flux push artifact ghcr.io/org/app-config:latest --path ./app-deploy

On the cluster you would reconcile the app with:

apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: OCIRepository
metadata:
  name: my-app
  namespace: apps
spec:
  interval: 10m
  url: ghcr.io/org/app-config
  ref:
    tag: "latest"
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: my-app
  namespace: apps
spec:
  targetNamespace: apps
  interval: 10m
  path: "./app-deploy"
  prune: true
  sourceRef:
    kind: OCIRepository
    name: my-app
  wait: true
  timeout: 2m

Ref: https://github.com/fluxcd/kustomize-controller/pull/684