Please also build docker for arm64 for cloud deployments

[DataCerealz]

Hi, this is a feature request.

We tried running MLflow Models with Kserve and ran into a variety of errors. Turned out mlserver was the issue - currently we run all workloads on AWS EC2 instances with graviton processors (arm64). The mlserver image pulled by kserve was not compatible.

Since a lot of cloud providers offer compute resources cheaper on their own arm64 chips; I think it would be great if mlsever would support arm64 as well.

Steps we took to solve it for now:

• clone this repository.
• Replace Dockerfile line 66 with wget "https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERSION}/Miniforge3-${MINIFORGE_VERSION}-Linux-aarch64.sh" \
• ran export DOCKER_DEFAULT_PLATFORM="linux/arm64/v8"
• build the image
• overwrite the default kserve mlserver image with our newly compiled image

Since then everything works as expected; so I'd assume this might be a small feature request.

[sakoush]

@DataCerealz This seems sensible thanks for pointing it out. Are you willing to contribute the suggested change as a PR please? We encourage contributions from the community.

[DataCerealz]

Thanks for the fast response! I would love to contribute and hope to get to it this week. It seems like a straight forward enough fix since I have already done it with a fork. The only thing that might take me a bit more time is to test the image properly. I originally got a warning for a faulty LD_LIBRARY_PATH during build which I ignored since CUDA wasn't relevant for our use case anyway; but for a proper pull request everything should work.