Support environmental variable or Dockerfile's ARG in Singularity recipe

[bbarker]

Version of Singularity:

2.5.1

Expected behavior

If BASEIMG is an environment variable exported where singularity is run, then it should be expanded in the Singularity recipe in all locations. In particular, this is obviously not done for From::

Bootstrap: docker
From: ${BASEIMG}

Actual behavior

When singularity build is run one gets a rather confusing Authentication error, but --debug helps:

VERBOSE Obtaining tags: https://index.docker.io/v2/library/${BASEIMG}/tags/list
DEBUG GET https://index.docker.io/v2/library/${BASEIMG}/tags/list
DEBUG Http Error with code 404
ERROR Authentication error, exiting.

Steps to reproduce behavior

export BASEIMG="alpine:latest"

Then build a file that has

Bootstrap: docker
From: ${BASEIMG}

I attempted a workaround of adding the %environment section before From:. Not sure that would even help, but it appears to not be allowed.

This touches on the more general problem (possibly) that build time environment variables (ARG in Dockerfile parlance) may not be supported, though I suppose they could easily be if:

1. any exported environment variable is expanded in the Singularity recipe
2. only environment variables listed in %environment are propagated to the singularity image.

So environment variables meating requirements 1 and 2 would be similar to ENV in Dockerfile, and those matching only 1 would be similar to ARG. This is still a bit different perhaps than how Dockerfile does it, but close enough, I think, to get things done. Another difference is that docker build passes these in via command line arguments to docker build, rather than as pulling in environment variables, but I don't think it is necessarily advantageous to split hairs on this behavior at this stage.

[bbarker]

FWIW, here is the discussion that led up to this: https://groups.google.com/a/lbl.gov/forum/#!topic/singularity/lu1ic7JKQj4

[bbarker]

And also: https://github.com/singularityhub/singularity-cli/pull/40 Turns out I was a little too lenient in my review of the PR, as I hadn't actually had a chance to test the singularity build of the output Recipe (but now that I have, or almost have, a reasonable singularity dev environment ready to go, that should be less of a problem).

[bbarker]

Double quotes around environment variables aren't erased either, for instance, in the %files section, "./${DISTRO_INSTALL_CMDS}" gives rise to Copying '"./alpine_install_cmds.sh"' to ... resulting in a copy error.

[md-k-sarker]

Any update on this issue?

[luator]

+1, it would be great to have some way of passing arguments to the build, so that we don't need to hard-code file paths from the host system in the definition file.

[carterpeel]

Hello,

This is a templated response that is being sent out to all open issues. We are working hard on 'rebuilding' the Singularity community, and a major task on the agenda is finding out what issues are still outstanding.

Please consider the following:

1. Is this issue a duplicate, or has it been fixed/implemented since being added?
2. Is the issue still relevant to the current state of Singularity's functionality?
3. Would you like to continue discussing this issue or feature request?

Thanks, Carter

[vsoch]

Just ran into this today, still needed!

[luator]

Yes, this is still something, I would find very useful.

[pedroalvesbatista]

@bbarker We're looking into the issue carefully, soon will bring to community and discuss ways to better solve as well address this. Thankyou for keeping the interest in the subject.

[kmuriki]

Old Singularity repo issues are now closed after copying into the new Apptainer repo https://github.com/apptainer/apptainer/issues/626