r-debug icon indicating copy to clipboard operation
r-debug copied to clipboard

Prepare images for arm64 compatibility

Open krlmlr opened this issue 1 year ago • 6 comments

I'm struggling to run the images (or R, for that matter) on my M2 in a virtual machine using Rosetta. I'm experimenting with arm64 images in https://github.com/cynkra/r-debug. These Dockerfiles have worked for me on arm64, chances are that they will continue to work on amd64.

Are you open to building multi-arch images in this repo, and perhaps to hosting them (also) on the GitHub container registry? What does CI/CD for these images currently look like?

krlmlr avatar Jun 03 '23 20:06 krlmlr

Yes, I think it's a great idea to build multi-arch images. I don't know anything about the GitHub container registry, but if the images can be hosted there instead of (or in addition to) Docker Hub, that would be good.

The CI is a daily build, which happens on a computer in my office. I did this because the builds took too long on the Docker build servers and timed out. It's possible that they could be built in GitHub Actions as a multi-step build, but I haven't tried.

The build machine is Intel, so it won't be able to build ARM images.

If you want to make these things work, I can give you access to this repo.

wch avatar Jun 03 '23 22:06 wch

I have a successful build of r-devel (not r-debug) on arm64 on GHA: https://github.com/cynkra/r-debug/actions/runs/5167865060 . This build runs for almost six hours, the limit for GHA. (QEMU is slow, it would be faster if we had arm64 workers. Soon, or perhaps self-hosted.)

For now, to make this work for the entire suite of images, I think we need multi-step builds or a build matrix.

How do you feel about having multiple smaller images, one for each flavor?

krlmlr avatar Jun 04 '23 15:06 krlmlr

Update: I managed to run the original x86_64 version of your image on my M2. Blog post following, unless this is old news and I was missing something.

Having arm64 images will help in the medium to long term (much faster, perhaps detecting slightly other types of errors), but lower priority for me now. I can help with moving the build to GHA to ease the burden on your office machine and to have CI/CD.

krlmlr avatar Jun 04 '23 17:06 krlmlr

GHA for the original image now running at https://github.com/cynkra/r-debug/actions/runs/5170275732/jobs/9313080747 . Let's see if it finishes in under 6 hours.

Would MAKEFLAGS=-j2 help, or is it already specified in the build instructions?

krlmlr avatar Jun 04 '23 17:06 krlmlr

How did the build go?

IIRC, it already uses -j4.

wch avatar Jun 06 '23 20:06 wch

The arm64 build for r-devel finished in just under 6 hours, we may want to split that image further (or wait for arm64 builders to become available). Emulation is slow.

The entire amd64 build takes 90 minutes on GHA. See https://github.com/cynkra/r-debug/tree/latest . GHCR packages are created automatically (using the GitHub token), for Docker you'd need to add secrets and tweak the workflow YAML.

Successful build: https://github.com/cynkra/r-debug/actions/runs/5173276938/jobs/9318400873

krlmlr avatar Jun 07 '23 05:06 krlmlr