Include kustomize file to deploy to kubernetes

[GerardoGR]

From the documentation https://github.com/spiffe/spire/blob/main/doc/using_spire.md:

No official Helm chart, Kustomize file, or custom resource operator is available for SPIRE but Quickstart for Kubernetes includes a basic set of Kubernetes YAML files for testing SPIRE Server and Agent

Does that mean that there is no plan to include a kustomize file for the spire server, agent and possibly the k8s-workload-registrar?.

I have already a basic implementation of this that I can submit as a PR if anybody is interested.

[evan2645]

Hi @GerardoGR, thanks for this issue.

There is some interest in publishing Helm charts for SPIRE (https://github.com/spiffe/spire/issues/2652), but as noted in the documentation we still don't provide anything "official"

How are kustomize files normally distributed and consumed? Many of the same questions from https://github.com/spiffe/spire/issues/2652 apply here too

[GerardoGR]

Hi @evan2645 thank you for following up on this and referencing relevant issues!

According to kustomize's official documentation, the way to distribute kustomize files is by having a dedicated GitHub repository with only those files, cloning/forking that repository locally, and reference that as the base, e.g:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
  # kustomize/base is just an example directory inside the spire repo
  - ./path/to/cloned/spire/repo/kustomize/base

It should be possible though, to also reference a remote commit-hash and a directory inside the repo and use that as the base, e.g:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
  # ref=master is the git reference
  - github.com/spiffe/spire//kustomize/base?ref=master

Regarding the questions you mentioned in https://github.com/spiffe/spire/issues/2652#issuecomment-1004334020, I take these as more or less "action-items" for this kustomize feature/issue:

A new GitHub repository in the SPIFFE organization ~Helm chart itself~ Kustomize files, with documentation and examples GitHub Actions based integration tests that run nightly and on PRs Two maintainers

I can have a look to prepare a new repo for the spire kustomization files, send that as a PR here (or to the new repo inside the SPIFFE organization) including the github action to run a kind cluster integration test (borrowing the idea from https://github.com/spiffe/spire/issues/2652#issuecomment-1004350568 @sudo-bmitch). If there is already an integration test suite, could you share me a link with them? otherwise I would imagine some simple "smoke-tests" (e.g checking that the spire-agent/server come up alive) might be enough. What do you think @evan2645 ?

P.S I'm happy to help to maintain the kustomize repo :)

[GerardoGR]

Hi @evan2645 ! I've created a new public repository: https://github.com/GerardoGR/spire-kustomize that can work as the foundation for the kustomize manifests. It also includes an end-to-end test (based off the Kubernetes (K8S Workload Registrar "webhook" mode) integration test) that verifies that the spire-server, k8s-workload-registrar and spire-agent manifests work as intended. If you think this is good to go we could then move this repo into the spiffe organization, e.g spiffe/spire-kustomize.

[GerardoGR]

Friendly reminder: Would the above repo work for the spiffe org?

[rturner3]

Hey @GerardoGR, sorry for the delayed response. We really appreciate the effort you made to author Kustomize templates for the project. Providing a seamless experience for running SPIRE in Kubernetes is definitely a goal of the project.

In the past couple years, we have experienced significant community demand for project-sponsored Helm Charts. We have since made the strategic decision to unify on Helm Charts as the supported Kubernetes deployment templates for the project.

Before adding new projects to the SPIFFE organization, we always want to make sure there is a significant community demand and enough people committed to maintaining the project. We feel that it can be challenging to keep several versions of deployment templates in sync with the latest configurations in the project over time, and we want to make sure that users continue to have working deployment templates over time that are well documented.

All that being said, we have decided to close this issue out for now. If we hear significant community interest in project-sponsored Kustomize templates in the future, we are open to revisiting the idea of official Kustomize templates, but for now we recommend using the official Helm Charts.

[GerardoGR]

Hi @rturner3

Thank you for coming back to me. I understand the reasoning behind the decision and it sounds to me like like a sane path forward for SPIFFE as an organization.