Add support for ppc64le and s390x architectures and distroless builds

[prb112]

What type of PR is this?

/kind feature

What this PR does / why we need it:

Kubernetes supports multiple architectures - amd64 arm arm64 ppc64le s390x. In order to use scheduler plugins on ppc64le or s390x, I'd like to build the scheduler-plugins container image with support for s390x and ppc64le in the sig.

This would be in addition to the already provided x86_64,arm64 container image. This request is similar to https://github.com/kubernetes-sigs/scheduler-plugins/issues/113

Which issue(s) this PR fixes:

Fixes #745

Special notes for your reviewer:

We are using the scheduler-plugins on multiple-architectures.

Does this PR introduce a user-facing change?

adds container images for the  ppc64le and s390x

[k8s-ci-robot]

Welcome @prb112!

It looks like this is your first PR to kubernetes-sigs/scheduler-plugins 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/scheduler-plugins has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. :smiley:

[k8s-ci-robot]

Hi @prb112. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[netlify[bot]]

✅ Deploy Preview for kubernetes-sigs-scheduler-plugins canceled.

Name	Link
🔨 Latest commit	9c1b3fbebe870e18d811e12f69258c12dd3e8b9f
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-scheduler-plugins/deploys/66a129b0f5c49d00083cbd26

[prb112]

We're going to verify the commands (I meant to put as WIP)

[ffromani]

/ok-to-test

[ffromani]

I'd be in favor of this support, but we need to check we have available machines in CI

[prb112]

I'd be in favor of this support, but we need to check we have available machines in CI

Thank you @ffromani

[ffromani]

/assign

[prb112]

Hey @ffromani and team, @Chandan-Abhyankar and I have completed our testing. I have removed the work in progress label. We look forward to your review and comments and next steps. Thank you, Paul

[Huang-Wei]

@prb112 how urgent you'd like to merge this PR? I'm thinking of support distroless image building so that we have a unified image that works for all archs:

• amd64
• arm64
• s390x
• ppc64le

[prb112]

@prb112 how urgent you'd like to merge this PR? I'm thinking of support distroless image building so that we have a unified image that works for all archs:

Hi @Huang-Wei I'd love to get this sooner than later - our testing starts in 1-2 weeks. We have a custom build. As long as the distroless arrives in a month, it would work with our usecase too. Thanks, Paul

[Huang-Wei]

As long as the distroless arrives in a month, it would work with our usecase too.

Usually we cut releases every 3~4 months, so it's possible to get the distroless build code in, and cut a 1.29 release in early July. Does that sound feasible?

(BTW: even we proceed with this PR as a temporary solution, we will still have to wait until early July for the new release.)

[prb112]

As long as the distroless arrives in a month, it would work with our usecase too.

Usually we cut releases every 3~4 months, so it's possible to get the distroless build code in, and cut a 1.29 release in early July. Does that sound feasible?

(BTW: even we proceed with this PR as a temporary solution, we will still have to wait until early July for the new release.)

Hey @Huang-Wei that sounds a-OK. Is this something we could help complete? Is there a scope of work / containerfiles that would need to be changed? Many thanks, Paul

[Huang-Wei]

Is this something we could help complete?

@prb112 definitely, fell free to self-assign to work on it.

Is there a scope of work / containerfiles that would need to be changed?

Other Kubernetes sub-projects have some existing examples. I usually use cs.k8s.io to search - here one example (keyword: gcr.io/distroless/static:nonroot)

[prb112]

Hi ... apologies, which issue #, and then I'll assign. Thank you, Paul

[Huang-Wei]

I created #756 to track the distroless build work.

[prb112]

/wip

[prb112]

Hi @Huang-Wei Hope all is well. The distroless build details are included in this PR. We've confirmed it builds the correct images. Please let us know if you have any review/comments. Thanks Paul

[prb112]

Hi @Huang-Wei checking in to see what you may have as next actions, best wishes, Paul

[Huang-Wei]

@prb112 do you need some help on this? (I'm starting to prepare cutting a release this week)

[prb112]

@Huang-Wei thanks for the reminder. I'll update now

[prb112]

/test pull-scheduler-plugins-unit-test-master

[prb112]

Hi @Huang-Wei

For podman, you can create the images like this:

❯ BUILDER=podman RELEASE_REGISTRY=quay.io/pbastide_rh/openshift-demo PLATFORMS=linux/amd64 make release-image

For docker, you can create the images like this:

❯ RELEASE_REGISTRY=quay.io/pbastide_rh/openshift-demo PLATFORMS=linux/amd64 make release-image

This creates the manifest-listed image for a single architecture, by default it'll create an image for linux/amd64,linux/arm64,linux/s390x,linux/ppc64le.

You'll see the built images at:

• https://quay.io/repository/pbastide_rh/openshift-demo/controller?tab=tags
• https://quay.io/repository/pbastide_rh/openshift-demo/kube-scheduler?tab=tags

Note, I had to use Podman due to company restrictions, and have attempted to keep true to docker.

Thanks, Paul

[Huang-Wei]

For docker, you can create the images like this:

Sounds good. Thanks!

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Huang-Wei, prb112

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [Huang-Wei]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[prb112]

Thank you very much @Huang-Wei