spire
spire copied to clipboard
spiffe
Create multi-arch image
The result of CI/CD process should be a multi-arch image consumable by both ARM64 and x86 arch.
My plan is to use docker buildx for multiarch support. ref: https://github.com/kubernetes-csi/node-driver-registrar/pull/63
Andrew Harding: A quick note that SPIRE server requires CGO for the sqlite3 dependency. SPIRE agent also has a CGO dependency on some arches (I think maybe only darwin?)
I found that using these link flags fixed the sqlite3 dependency. server CGO_ENABLED=1 go build $$(go_flags) -ldflags $$(go_ldflags) -linkmode external -extldflags -static
agent, simply adding CGO_ENABLED=0 worked to compile a static agent
Supporting multiple archs from a single image would be awesome. I don't know anything about buildx but I'm excited to see what you come up with.
My main concerns are that we compile with CGO for the server (i think agent is ok w/o CGO, at least on linux) and that we leverage the existing build scripts as much as makes sense (i.e. the Makefile) but it sounds like both of those points are reasonably addressable?
If you have specific questions about the Makefile or want to banter around ideas, please feel more than welcome to reach out here.
Would goreleaser be accepted to cross compile for different platforms and architectures? That would make things a lot easier. Goreleaser in the backend does support buildx when running on the matrix. A nice benefit of goreleaser is that it also integrates with Sigstore so we can also start signing the releases and add SBOMs and build provenance. 😄
Hi, we (w/@marcofranssen) are willing to work on this, can anyone assign it to me?
/cc @anvega @azdagron ☝️
Thanks for raising your hand, @developer-guy and @marcofranssen, to take a look at this. I've gone ahead and made the assignments as well as updating the labeling on the issue.
We suspect that switching to goreleaser might not be without its challenges and are not quite sure what the scope will be to get everything working with feature parity and compatibility with the way we currently create/publish artifacts. We suspect there might be some gotchas that you encounter during the journey. As such, as part of taking this work on, do you think you'd be able to do a quick spike into what it would take to deliver a goreleaser based pipeline and provide an update on this issue with a light plan of action that the maintainers can review? That would certainly help us understand the scope of required changes that you plan to address.
Thanks again!
I have started a draft PR. #3607
In this PR I tried to stick with what is there today.
The problem however is the way images are shared across workflow jobs.
xbuild uses a cache within a builder. Meaning the build images are not available to the docker runtime causing the current failures. This would have been same issue with goreleaser.
I'm now searching if there is a way to share this builder cache via github cache action. That would allow us to simply rebuild the images to push them or to load them for a single architecture for the integration test from the existing build.
Should we still export images on a disk as tarballs? Because since we use docker buildx, the docker exporter does not support exporting manifests lists yet. So, my suggestion would be to push those images into ghcr.io otherwise we need to find a workaround for multi-arch images to save them on a disk. Does that make sense? @azdagron @marcofranssen
IIUC, the reason behind outputting images to a disk is that we want to test them first; then, if those images pass the test, we could publish them to a registry. But the docker archive format is not suitable for doing this, and we should use OCI image layout format, which supports saving multi-arch images on a disk but requires a lot of extra work. It will make the release workflow more complex than today as well.