jamespfennell
chore: add multi-platform build support to Dockerfile
This should help with #153 --
docker.io/milas/transiter:upstream
is a multi-platform arm64 / amd64 image built from HEAD:
docker buildx build --platform="linux/arm64,linux/amd64" -t milas/transiter:upstream .
Takes 42 seconds from scratch on my (admittedly fast) laptop.
fyi the main Docker build won't do the codegen verification anymore. If you want that to always happen implicitly, we can change around the targets so that it's FROM verify-codegen AS build to force that to run first, for example. It makes "clean" builds take slightly longer because of the tools install, but they'll then be cached afterwards.
Another alternative is to use a docker-bake.hcl and swap CI to the Bake Action: that'd make it possible to have both targets run as part of a single invocation, sharing layers etc.
Apologies if I went a bit overboard on making changes to the Dockerfile -- I'm totally happy to dial it back and keep it almost the same as it was but with a sprinkling of --platform and GOARCH. I know I also bled into the justfile 😅
Since apparently this is a habit of mine & I know I've used several likely unfamiliar features, this might be useful context https://github.com/kubernetes-sigs/headlamp/pull/1077#issuecomment-1506427051
You are of course welcome to make any changes or simply use this as inspiration if you'd like to handle it yourself -- consider my changes CC0 so won't be at all offended if you copy-pasta from it.
Also, not that it's surprising, but I have successfully run the image on an arm64 machine though I haven't connected it to DB yet since I found out that docker.io/postgis/postgis is missing arm64 builds as well, so that's next on the list 😉
Hey! Sorry for the late response, I've had a very busy ~6 weeks and haven't had mental bandwidth to respond to GitHub things.
Thanks for the PR!
Two comments:
(1) I think I'd prefer the build to require the code gen to run. My concern here by the way is security. Right now in PRs it's not necessary to review the generate files because we can assume they are created correctly from the proto and sql files. However if we don't verify the code gen then we actually would need to start inspecting the files to ensure that the PR isn't malicious.
(2) Do you think that you could add very tiny comments to the Dockerfile explaining why things like --link and --mount=type=cache,target=/go/pkg/mod are needed. Right now the Dockerfile is self-explanatory, but with this PR there's stuff in there that is hard to explain unless you're a Docker expert. :)
Thanks for this and sorry for the delay!
Btw I guess in theory the lack of an arm64 Postgis Docker image isn't a blocker - you can always point arm64 Transiter at managed Postgres database on a different host :). But would be cool if you could run the whole thing on an arm64 machine.
Absolutely no problem / worries on the delay (that's what forks are for 😉)
I will try to find some time to make those changes.
RE: Postgis for ARM64, hopefully there are official images one day, but for now, it's reasonably straightforward to build your own (needs QEMU/emulation if building from an AMD64 host, which will probably be very slow, building on native ARM64 host is ideal):
docker buildx build --platform=linux/arm64 https://github.com/postgis/docker-postgis.git#master:17-master
Several months later -- check out https://github.com/jamespfennell/transiter/pull/155/commits/3df01aba8e5e31396c0d1e868ab3edecebfd7a07 at your leisure:
- Build stages use files from
verify-codegen, i.e. after having generated the files from scratch and then successfully compared them to the ones in-repo- This makes CI Docker builds "self-verifying" and caches very well for local incremental builds
- Comments to explain some of the more nuanced Dockerfile bits
- Correct the indexing of the overall comments which I evidently messed up