postgres-operator
postgres-operator copied to clipboard
CrunchyData
ARM support status?
Hello! AWS Graviton2 instances are becoming more tempting and after testing Postgres on them, we'd like to deploy ARM nodes. I did found an article on your site regarding ARM testing https://info.crunchydata.com/blog/postgresql-benchmarks-apple-intel-macbook-pro-2011-2019 but no mentions of ARM here.
So, do the currently released images support ARM? I mean the whole suite, not just the operator. They seem arch-agnostic to me, but confirmation would be better.
I've used the operator on x86_64 and wanted to see how a deployed the operator in a hybrid arm64/x86_64 cluster would work out as I ported containers. I have to hot-patch the deployments with a nodeSelector. And that sort of works. Where I get hosed is with jobs, because they are immutable.
Running on arm64 would be sweet.
It would be beneficial for small environments such as raspberry pi based clusters to support arm64.
I would be willing to take a stab at a PR for this if someone could outline the approach. In particular, I'm thinking I'd just need to add a docker-buildx value for the Makefile's IMGBUILDER parameter? Or maybe just change the behavior of the IMGBUILDER=docker so it uses docker buildx build instead of vanilla docker build? Although I'm not sure how to configure the CI job that actually invokes make--it doesn't seem to be in .github/workflows/.
Does the underlying https://github.com/zalando/spilo image need to be made arm64 compatible first?
https://github.com/CrunchyData/postgres-operator/blob/v4.7.1/Makefile#L142
That amd64 is a problem there I believe.
I've used the operator on x86_64 and wanted to see how a deployed the operator in a hybrid arm64/x86_64 cluster would work out as I ported containers. I have to hot-patch the deployments with a nodeSelector. And that sort of works. Where I get hosed is with jobs, because they are immutable.
I also have a multi-arch cluster. I probably have 95% of things working truly multi-arch, the rest using NodeSelector as above. In cases like these, where an operator is spawning jobs, and there is no available place in the CRD to add a NodeSelector, I've used the PodNodeSelector admission controller with an annotation on the namespace to limit my postgres-operator namespace, and that of the actual db to an arch of amd64. -- see https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/
Did you have to manually rebuild stuff? Is what you have open source/present somewhere?
Any update on this one? A couple of years on with arm64 becoming more prominent I just tried spinning up the operator using the HELM chart on ARM64 cluster and the result the dreaded symptoms (see below) of an unsupported architecture.
$kubectl logs pgo-6986ddf484-d2pn8
exec /usr/local/bin/postgres-operator: exec format error
In the mean time, would you consider adding a nodeSelector for those architectures that are supported? That would at least help those of us who have mixed architecture clusters.
I've been doing this manually for my clusters, but it would be great if it worked out of the box.
Any update on this one? A couple of years on with arm64 becoming more prominent I just tried spinning up the operator using the HELM chart on ARM64 cluster and the result the dreaded symptoms (see below) of an unsupported architecture.
$kubectl logs pgo-6986ddf484-d2pn8 exec /usr/local/bin/postgres-operator: exec format error
Same here. I don't have a multi-arch cluster, only ARM64 nodes. Gutted to find this operator isn't compatible with my cluster.
We're setting up a multi-arch cluster and are also a bit troubled by the lack of ARM support and lack of built in multi-arch support via nodeSelector.
Could anyone recommend alternative Postgres Operators which support ARM or multi-arch out of the box that could be considered until this gets resolved?
I'm also still anxiously awaiting ARM64 support. In the meantime though @dbackeus , if you have a multi arch cluster but want to run crunchy only on the amd64 nodes, I've successfully used node affinity selectors in the cluster yaml definition. This works for me because I am not using any of the scheduled jobs/backups , so this at least gets the core cluster up and running on amd64 nodes. I looked at and tested many alternatives, and none were as robust as crunchy was to node failures, etc, so this works for me for now, but I actually have some ARM64 nodes arriving soon that would be faster than my amd64 nodes, so would like this resolved soon as well.
Is there anyone from Crunchy (or just a contributor who knows a bit more than me?) who can advise if there are open work items for this? We are happy to dedicate time for the OSS components that may need work/support to make this work on ARM64. Frankly, the savings we will get in the first month would pay for our labor investment. Happy to direct some resources this way if we can get even a basic point in the right direction!
+1 for this issue. Basically it blocked everyone will ARM node to try postgres-operator, is there a branch we work on for getting an ARM image?
I've built arm images of crunchy before; it's fairly straightforward with the instructions. Perhaps it's something that can be added as a GitHub action ?
Decided to try crunchy products for the first time, was disappointed by the lack of ARM support, which, it seems, has been basically mandatory for all serious software for several years now. Until this support appears, I cannot use this operator, because my cluster is fully arm64 and this won't change
I've built arm images of crunchy before; it's fairly straightforward with the instructions. Perhaps it's something that can be added as a GitHub action ?
Can you please tell us procedure you followed
@jelmer , can you guide us how can we build crunchy arm images in local, can we modify the makefile and achieve this.
@jelmer , can you guide us how can we build crunchy arm images in local, can we modify the makefile and achieve this.
It's been a while since I've used the Crunchy operator, but I believe it was just a matter of replacing amd64 with arm64 in a few places in the makefile / overriding those variables from the command line, and then following the regular process for building images as described on https://access.crunchydata.com/documentation/crunchy-postgres-containers/4.5.0/contributing/building/
ARM support was introduced in the latest release (5.4.0). See announcement here: https://www.crunchydata.com/blog/announcing-crunchy-postgres-for-kubernetes-5-4
I guess this issue should be closed?
ARM support was introduced in the latest release (5.4.0). See announcement here: https://www.crunchydata.com/blog/announcing-crunchy-postgres-for-kubernetes-5-4
I guess this issue should be closed?
I don't know what kind of ARM support they added, but at the moment the latest version 5.4 doesn't work on arm64 processor. The good old “executable format error”.
It is a pity that such an interesting and convenient project still does not support ARM. Looks like I'll have to rebuild it manually if I want to try it with my arm64 cluster
"Currently ARM images are available for crunchy software subscription customers only and not available as part of our developer program. We may change this in the future, but at this time we're working closely with customers to ensure a quality experience as this architecture before we explore making more broadly available in the developer portal."
https://groups.google.com/a/crunchydata.com/g/postgres-operator/c/roplBujSiDU
😳
This should be a warning to anyone considering using their products in production, paid or otherwise. They explicitly refused "free" (community) labor to accomplish this, and instead did it internally, possibly just by copying the community answers, and are now charging for it..
As pointed out here in places, Crunchy Postgres for Kubernetes is our commercial product. It is composed not only of the Postgres Operator but also corresponding components and container builds that go through various certification and testing. For many of our customers we have specific commitments in how that software is built and tested. Part of that is being built on specific hardware.
Because of our commitment to our customers we could not simply grab a cloud instance and build on it. As a result it took us over a year to procure the needed hardware to build our ARM containers, the demand for ARM hardware is quite high and we worked through multiple vendors before finally having an amazing experience with Ampere.
At this time those builds are only available to customers of Crunchy Postgres for Kubernetes (in version 5.4) because we want to work closely with those using them to ensure high quality and a good experience. We are evaluating the prospect of making these ARM builds available within the Developer Portal as part of the Crunchy Data Developer Program, but don't have a set timeline for that yet.
PGO community members have commented elsewhere in this thread regarding how it may be possible to build PGO for ARM infrastructure. We have not reviewed / validated those comments, but perhaps they are useful to individuals looking to build PGO ARM infrastructure. Our only request in doing so is that you abide by the Apache license applicable to the source code and project policies.
Finally, we have recently updated our documentation - which includes FAQs to address the intersection between PGO (the open source project) and Crunchy Postgres for Kubernetes (the Crunchy Data commercial product): https://access.crunchydata.com/documentation/postgres-operator/latest/faq
This is where community involvement would have assisted Crunchy. I have spare ARM hardware, I'm sure others do also. There are numerous ways to build ARM containers without procuring hardware (github actions + qemu).
That being said, if I can't evaluate open source versions of licensable products that my customers may want, I won't recommend those products to my customers. At this point, I consider Crunchy's operator a couple years late, and a few dollars too much.
-- Curtis Ruck
On Tue, Aug 8, 2023 at 2:04 PM Craig Kerstiens @.***> wrote:
As pointed out here in places, Crunchy Postgres for Kubernetes is our commercial product. It is composed not only of the Postgres Operator but also corresponding components and container builds https://access.prodtest.crunchyds.com/documentation/postgres-operator/latest/overview that go through various certification and testing. For many of our customers we have specific commitments in how that software is built and tested. Part of that is being built on specific hardware.
Because of our commitment to our customers we could not simply grab a cloud instance and build on it. As a result it took us over a year to procure the needed hardware to build our ARM containers, the demand for ARM hardware is quite high and we worked through multiple vendors before finally having an amazing experience with Ampere.
At this time those builds are only available to customers of Crunchy Postgres for Kubernetes (in version 5.4) because we want to work closely with those using them to ensure high quality and a good experience. We are evaluating the prospect of making these ARM builds available within the Developer Portal as part of the Crunchy Data Developer Program, but don't have a set timeline for that yet.
PGO community members have commented elsewhere in this thread regarding how it may be possible to build PGO for ARM infrastructure. We have not reviewed / validated those comments, but perhaps they are useful to individuals looking to build PGO ARM infrastructure. Our only request in doing so is that you abide by the Apache license applicable to the source code and project policies.
Finally, we have recently updated our documentation - which includes FAQs to address the intersection between PGO (the open source project) and Crunchy Postgres for Kubernetes (the Crunchy Data commercial product): https://access.crunchydata.com/documentation/postgres-operator/latest/faq
— Reply to this email directly, view it on GitHub https://github.com/CrunchyData/postgres-operator/issues/2035#issuecomment-1670066773, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK4HYO4IDYNPA2IL3ERR23XUJ5SXANCNFSM4TPYV2EQ . You are receiving this because you commented.Message ID: @.***>
There are numerous ways to build ARM containers without procuring hardware (github actions + qemu).
Not only that, but Docker also supports multiarch builds out-of-the-box, anyone can easily build arm64 image on x86 host
That's said, both alternative products (Zalando's Operator and Cloudnative-pg) are supporting arm64 without any kind of paywall
Thus, the path is clear, and it does not lead to CrunchyData. I don't consider companies that pretend to be open source, only to close even basic things behind a paywall.