Allow sealed-secrets to be installed as a kustomize `base/resource`

[saviogl]

With kustomize allowing composition of multiple kustomize layers it would be great to have sealed-secrets to be able to be installed in such way.

Here is what it would look like from someone wanting to install sealed-secrets in their cluster:

# kustomization.yaml
bases:
- github.com/bitnami-labs/sealed-secrets//path/to/kustomize/dir?ref={git tag}

That is a very handy way of installing the project as well as keeping up with new releases. It would only require as part of the release process to have a folder that has a kustomization.yaml file referencing all necessary resources.

A few projects have gone that route and it has streamlined things for my team... The latest example that we saw of that was argocd

[mkmik]

uh, I wonder why didn't they allow a simple url like https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.9.4/kustomization.yaml to work.

apparently they use https://github.com/hashicorp/go-getter which treats github.com specially (or perhaps I didn't find the right way to encode such an url?)

[mkmik]

I tried go-getter https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.9.4/controller.yaml /tmp/foo and it works, so perhaps kustomize is compiled with an old version of it?

[mkmik]

we have an automated release process that gets kicked off when we tag a release e.g. v1.2.3. The build automation builds a docker image, tags it and when all the tests are green it pushes this tagged image to the docker registry. Then we generate a controller.yaml file that references such a tagged docker image and publish it in the github artifacts.

I'd love sealed-secrets to be more friendly to kustomize, so I'm willing to do something about it.

However, if I understand correctly your proposal, this would:

1. require us to change our release process to include a commit that updates the tag to match the next release and only after that commit we can tag a release. I'm concerned about us forgetting to do this step.

2. everybody consuming sealed-secrets using this method will effectively have kustomize clone the whole sealed-secrets repository just to fetch a few files (I checked, on my machine it checks out the whole repo in a place like $TMPDIR/kustomize-379970711)

Is there a better way?

[mkmik]

https://github.com/kubernetes-sigs/kustomize/blob/master/examples/goGetterGeneratorPlugin.md#remote-sources

Kustomize supports building a remote target, but the URLs are limited to common Git repository specs.

[saviogl]

@mkmik Given that the docker image follows the same tagging strategy as the repository does, what if as part of your release pipeline you start updating a fixed location (manifests/...) that contains all manifests for the given version (ref: tag/commit) and continue the release process as is but tagging th release as pre-release to indicate that is not production ready yet. After the image has been built, tested and published besides publishing the controller.yaml to the github artifact you would also mark the pre-release as released.

Here's what the file structure could look like:

...
manifests
|-- controller.yaml
|-- kustomization.yaml
...

# controller.yaml
...
image: quay.io/bitnami/sealed-secrets-controller:v0.9.4
...

# kustomization.yaml
resources:
 - ./controller.yaml

I haven't had the time to dig through this project Makefile to better structure my proposition but I was thinking something along those lines. Does that help?

As for the second point haven't checked that behavior yet but we could hit the kustomize team about ways of improving that.

[mkmik]

The release process is triggered by creating the GitHub release, which creates a git tag with whatever commit is master's HEAD at the time. Pre-releases are no different, they are normal tags except that they are reported as pre-releases by GitHub and they are not considered when computing the "latest" release alias.

This means there is a chicken and egg problem: if I push a new commit during the release automation step it will be added in the master branch but the resulting files won't be visible at the v1.2.3 tag which was created before the release automation started.

It's technically possible to delete/recreate a tag but last time I tried that it confused the hell out of GitHub's release page.

[saviogl]

@mkmik Yeap, these files need to exist before the release gets created in GitHub then. Would be possible to add a step prior to the Github release creation to create/update those manifest files within the current process?. At the place that I work we made extensive use of semantic-release to automate github release generation workflow. We have projects that require building artifacts and updating/commit package versioning (package.json) prior to the release generation.

[mkmik]

yep, probably that's a reasonable approach given the current state of kustomize. It might be worth bringing this issue up with the kustomize team and see whether it would be possible to extend the remote build feature to support plain HTTP(s) urls (perhaps by requiring ?checksum= if they are concerned about hermeticity?)

[pikeas]

This would be great, the helm install works but kustomize would be even better!

[mkmik]

Glad to see interest on non-helm. Bumping priority for this.

[pinkertonpg]

If I understand the deprecation timeline of the helm/charts repo properly, the helm chart currently linked to in the README will be no longer usable on November 13th, 2020. We can always fork it and host it ourselves, but for a deployment as simple as this one kustomize would be much preferred.

[mkmik]

Ok. I'll bump the priority of this

[cwimmer]

Is there any update on this @mkmik ? I'm still looking for an elegant way to install sealed-secrets with Kustomize.

[mkmik]

Yes, I'm overhauling the release automation and it soon should be possible to include a pre-rendered manifest.

The next release will be cut with this automation

[mkmik]

Unfortunately this didn't make it in the v0.14.0 release. Moving to v0.15.0

[github-actions[bot]]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.