Environment variable for SingularityCE to look for image files

[HybridNM]

It would be nice to have a PATH-like environment variable where SingularityCE can look for images when no path is specified, so we can have multiple directories storing images.

So, let's call it SINGULARITY_IMAGE_PATH, is set to /home/user1/singularity/images:/home/user2/containers. The working directory is /home/user/work, which has no singularity images. You run a singularity exec/run/shell command, specifying just the name of the image, without a full path, and SingularityCE searches in all directories in SINGULARITY_IMAGE_PATH, along with the working directory.

This could cause conflicts if images in different directories in the env. variable share the same file name, but it shouldn't be too hard to find a workaround.

[dtrudg]

Hi @HybridNM - this is something that we have seen requested in the past, but my gut feeling is that it's something that..

1. Could be confusing to users. A lot of Singularity's user base isn't too familiar with how PATH works. E.g. we need to think about how it would appear if you consider an administrator setting SINGULARITY_IMAGE_PATH but a user following any of the many public guides that doesn't mention it.

2. Makes execution commands work differently to build and possibly other commands (which couldn't sensibly search a PATH).

3. Needs very careful though around exposing exactly what is happening when 'verify' command, or ECL are used, so there is not a chance of misleading a user about the execution of a trusted container.

4. Would have to consider situations such as nested execution of Singularity inside Singularity etc.

What I would generally encourage for proposals of features like this is a full explanation of the scenario in which it would be useful to you, and how you would want to set it up. Not so much a brief technical description of the path search, but detail around how you want to store the images such that a path search is useful etc. This can be beneficial to think about supporting workflows more completely and intuitively across commands, rather than adding inidividual features.

Cheers!

[HybridNM]

Hello, thanks for the lengthy response.

We use Singularity in our supercomputer clusters as a means to execute programs with overly complicated compilations or very strict dependencies. Our users may have containers in their home directories, for their personal use, while we sometimes create containers that are accessible by everyone and are stored on a single directory.

What we did for the time being was writing a wrapper script file that is executed the same way as a "singularity exec/shell", but internally searches for the container name within a PATH-like environment variable (in which the desired directory is automatically added when the module is loaded) and then calls the exec/shell command.

We understand the possible issues and development headaches so we won't push it further, since it's not really a big priority, but an addition we considered could be useful.

Thanks again for your attention, and best regards.

[dtrudg]

@HybridNM - thanks for the further details. We'll keep this in mind, for sure. If there's a way that appears to implement it consistently then it may be useful to others also.

What we often find in scenarios similar to yours is that people set up module files and a specific wrapper script, for each container that is provided centrally. It's good to be made aware of other ways people are handling this. Thanks!

[dtrudg]

I'm going to close this issue as it is not planned due to the complexities involved as discussed above.

We'll re-consider if there are additional requests for similar functionality.