linux/arm64/v8 Architecture images

[chris-cheshire]

Hi,

We are starting to be issued with M1 macs at our institution and we are getting docker errors like:

docker: Error response from daemon: image with reference biocontainers/biocontainers:v1.2.0_cv1 was found but does not match the specified platform: wanted linux/arm64/v8, actual: linux/amd64.

I guess this is because either the program is not available as an arm compiled executable, or that biocontainers doesn't build against arm as part of its build process.

Could anyone comment on this?

[chris-cheshire]

@osallou any comments on this? Happy to help implement but its quite a large change.

[osallou]

hi, we do not build (yet) for other archs than amd64. There is a plan to do so (at least arm), but for that we need to get some related hardware. We expect to have one arm server next year and add container builds for arm.

Be also aware that some tools may not build on arm....

[chris-cheshire]

Hi @osallou hows the state of ARM looking? You probably know that Apple have stopped selling intel macs and my laptop is on the way out so its becoming critical for everyone at the Francis Crick that this is sorted. Is there a way I can get involved and help move it forward?

[osallou]

Hi, we have no arm server unfortunately, and none planned for the moment... We may benefit arm servers from Amazon in future, but nothing sure for the moment. (@bgruening ?)

[bgruening]

Yes, the plan is to use some AWS infrastructure.

[chris-cheshire]

Ok I think this is going to become a real issue very soon. So many people test their nextflow pipelines locally on macos using docker - on an M1 this is currently impossible. Is there any timeline at all for this, is it a funding issue or development issue etc?

[bgruening]

It's more a priority and time issue. If the nextflow community needs it, they are welcome to contribute. E.g. adding support to bioconda-utils etc ...

[osallou]

Won t fix issue of arm based containers, need servers anyway

[chris-cheshire]

Yes I am very happy to contribute if that will help and I am sure I can get a few others as well if you need man-power. @osallou are you saying that AWS provide your server infrastructure for free but that they currently don't allow you to spin up ARM instances?

[osallou]

Aws will provide us compute resources among which we should have arm, but this is not yet the case

[chris-cheshire]

OK is there a timeline for when AWS will provide these?

[CharlotteAnne]

I'd just like to second the urgent need for this - use of MacOS is incredibly common in the scientific community. If there's anything I can do to help please let me know.

[osallou]

rebuilding everything for amr will take time anyway... but our main issue is to have an arm build server

[pditommaso]

I don't know much biocontainers build infra, but docker build can target both arm and amd architecture irrespective of the host chipset via emulator. Would that not work for biocontainers building as well?

[osallou]

Seems possible via buildx but on docker desktop only (windows/mac, and not free)

We build with docker ce, on linux

[pditommaso]

There are plenty choices nowadays to build containers such as podman, buildah, kaniko, etc but not sure about the multiarch support of these. I'll made some research about this.

[pditommaso]

There's a nice post showing how to use Buildah to build and push a multi-arch container images.

Also considering the changes in Docker license and the OSS nature of Biocontainers, think should consider moving away from Docker for container building.

[bgruening]

It has nothing to do with Docker. We are not building Containers here, we are building packages with conda. I think what we need is needed the following:

• get ARM/RISC support into bioconda-utils (take inspiration from conda-forge)
• implement an allow-list to be consumed by bioconda-utils where we add packages that should be compiled for ARM and/or RISC
• figure out against which ARM architecture we actually want to build - which is most compatible
• getting some computational build infrastructure in place to actually compile it

We have been working so far on the last item and a grant for the first three has not been granted in 2020. If anyone wants to help you are all very welcome.

[pditommaso]

I see, out of curiosity what's your build infra now? why something like GitHub action does not fit your needs?

[bgruening]

We can use whatever public CI system that is available. But it needs to support the correct (point 3 above) ARM architecture, ideally RISC as well - and all of that in a substantial amount.

Xref: https://github.community/t/ask-about-github-hosted-runners-arm64-platform-support/167483

[bgruening]

All of that said, I would like to manage expectations here ... this is nothing that we can do in a few weeks. (We are already trying this for a few years). And there are larger discussions to be taken in the overall scientific community.

• It's ecological and financial probably a bad idea to create ARM containers for all packages.
• The promises that we give to easily scale up a reproducible pipeline will not hold true anymore - at least to this degree. You very likely changing from M1/M2 to x86-64 based execution environments and with this, you are changing packages/containers.

There are good reasons we want ARM/RISC containers and large ARM HP clusters is one of them. OSX users can and maybe should for the foreseeable future use a VM. This is possible isn't it?

[osallou]

On mac, i think docker uses a microvm in fact, like on windows (but not 100% sure)

[osallou]

And even for non conda packages, other ones needs to be compiled on arm, which may not work for all tools, on a base distro arm compatible too with all their dependencies.

Building a tool made for x86 is not fully transparent for all tools

[pditommaso]

So, think we are discussing two topics here:

1. addressing the issue for Apple Silicon users, I don't think there's the need to rebuild Bioconda containers for that. As mentioned before, Docker made excellent work in this regard, it's just enough to use the CLI option --platform linux/amd64 or the env variable DOCKER_DEFAULT_PLATFORM=linux/amd64 to run any existing container on Mac M1 (with rare exceptions). It would be interesting to learn from @chris-cheshire if he thinks is a viable solution, and why not otherwise.

2. the building of Bionconda/Biocontainers targeting ARM architecture for production workload and optimization reasons. I argue there could be a strategic need to support all recipes, but maybe deserves a separate thread.

[chris-cheshire]

Thanks for everyones input! its great to get some dialogue going around this. @pditommaso I will do some testing around this running nf-core pipelines in test mode on an M1 mac this week and get back you all.

[chris-cheshire]

We have been doing some testing and it appears Docker Desktop has been upgraded as with the latest version the biocontainer images seem to run by default on nf-core pipelines without any extra config. We need to test some more pipelines but this is very promising!

[edmundmiller]

@delagoya Following up on this post Nextflow Summit. Thoughts on getting together a hackathon for a day or two to get started on the effort?

cc @drpatelh @maxulysse @ewels

[chris-cheshire]

I would be interested in participating in this

[delagoya]

@Emiller88 absolutely! I need to get over the next few weeks of conferences, so earliest I will be coordinating activities is mid-December