Pulling Singularity containers using Azure Batch C# SDK

Open fertinaz opened this issue 5 years ago • 1 comments

Feature Request Description

I'm building a cli tool that creates HPC pools on Azure. My main goal is to run Singularity containers pulled from ACR and run them inside the multi-node pools to execute complex (MPI, Cuda etc.) simulations.

I can perfectly run Docker containers. However, the same rationale failed when I tried to pull a Singularity container. In another discussion, https://github.com/Azure/acr/issues/411, I learned that Singularity containers are not supported on Batch Services.

As HPC people already know, Singularity solves some of the issues introduced by Docker on multi-node multi-tenant environments. So that would be nice to have feature.

Also, I'd like to know if there Is a potential workaround that you can suggest except using batch-shipyard which I already know.

Jun 23 '20 02:06 fertinaz

Batch currently does not support Singularity containers natively. At this time, you will need to perform the pull and execution mechanics yourself on top of regular Batch tasks (i.e., not container tasks). As you have mentioned, Batch Shipyard already supports Singularity container execution.

Jun 24 '20 16:06 alfpark