gloo
gloo copied to clipboard
solo-io
Build Official ARM64 Images - data plane pipeline
Is your feature request related to a problem? Please describe. Build official ARM images for the various gloo components
Describe the solution you'd like ARM compatible images
Describe alternatives you've considered N/A
Additional context Envoy now provides multi-arch images with arm64 support AWS ARM instances
Related Issues:
- [ ] https://github.com/solo-io/gloo/issues/9800
This should be re-opened as my PR did not solve this issue (it tracks official releases, which still has some work to be done)
Agreed, reopening! We have the makefile changes required to build ARM64 images of Gloo, but we don't have a CI/CD pipeline setup with the ARM worker machines required to build it yet. Our current pipeline is in GKE / Github Actions. We may need to integrate with AWS for this.
Further, for envoy-gloo there is at least one upstream dependency (glibc) which has no official arm64 support.
Indeed. I had to create the glibc image and the frovlad/alpine-glibc images for arm myself: https://github.com/CelsoSantos/alpine-glibc
I recommend Solo forks or recreates this repo in order to handle that
Yah depends on ask, we have the builds for Istio so should be able to get this theoretically if someone needs it.
It should be possible to use Docker's buildx feature to build multi-arch images without needing access to runners of different architectures.
Example doing this in GitHub Actions: https://github.com/tmobile/magtape/blob/b78170b5b020dd2f0a336d514e48fec5980227cb/.github/workflows/ci-image-build.yaml#L62
We have done this with Istio btw and found that there is much high CPU usage for TLS handshakes in ARM vs Intel/AMD. So for those looking to save money using Graviton it would depend on the number of TLS connections.
@chrisgaun would you have some more information to share? Could you eventually do a comparison for the same number of connections how that affect performance? (CPU/RAM/Network, etc). I'd be very interested in learning more about it
I know a lot of our use case behind this is more for local development so performance differences aren't as much a concern (still a concern if there's a drastic difference). A lot of our development teams use Apple laptops and those will be moving to arm64 as an only option before too long. We like to have a full k8s/gloo deployment in a local environment on the laptops and that's going to be impossible/difficult without upstream support for arm64
Ah yes. We have the same use case internally actually as many of us are on the M1s. We have good internal docs on this that we should add to our docs.
To add to the use case list: I want to use Gloo on my home lab k8s cluster as the ingress service. Doubt I'll see many TLS connections so that's not a major issue for me.
I'm going to second what @phenixblue stated. Exact same use case for us. Any updates on this issue being addressed?
Everyone. This seems to be a limitation on using Google cloud. They did not have ARM machines until very recently and even now it is very limited supply.
For Gloo Mesh we use an (expensive) self hosted AWS EKS cluster as a runner to build ARM images. Google says the ARM machines will be GA in "early 2023." Until then we can do one offs for customers potentially but more general pipeline still is a bit out.
You don't need ARM machines to build ARM images tho. All you need is docker with qemu to do a build or buildx to create the images. (It is slow to build...)
Any progress/plans on this? Currently, gloo-edge is the last amd64 workload in our clusters and (for cost optimization purposes) I wold love to have arm64 only.
From what I understand this would imply
- providing multi architecture OCI manifest (supporting
linux/arm64andlinux/amd64) forquay.io/solo-io/gloo - providing multi architecture OCI manifest (supporting
linux/arm64andlinux/amd64) forquay.io/solo-io/gloo-envoy-wrapper
Maybe I can help with a PR if this is not too complex :-), any pointers and hints are highly appreciated.
