run doesn't respect quotes and escapes the same way exec does

[mr-c]

Version of Singularity:

3.2.1

Expected behavior

$ singularity run docker://debian /bin/sh -c     'echo \"moo\"'
"moo"

Actual behavior

$ singularity run docker://debian /bin/sh -c     'echo \"moo\"'
moo

Steps to reproduce behavior

$ singularity exec docker://debian /bin/sh -c     'echo \"moo\"'
"moo"
$ singularity run docker://debian /bin/sh -c     'echo \"moo\"'
moo

[mr-c]

Any update on this @jscook2345 @ikaneshiro ? Thanks!

[WestleyK]

@mr-c, Sorry for the delay, looking into it now...

[WestleyK]

This seems to be a issue with the container (docker://debian). For example:

To start, lets build a container, with a known %runscript: (the %runscript gets executed when using the singularity run test.sif <args>, or ./test.sif <args>)

# First, make a known definition file
$ cat << EOF > test.def 
bootstrap: library
from: alpine:latest

%runscript
# Example 1. runscript can be as simple as executing all the arguments passed to it with quotes ("")
"\$@"

# Example 2. Or, it may not even have quotes
\$@

EOF

# Then build the container...
$ sudo singularity build test.sif test.def 

Now when we try to exec and run the container like you did, we get:

$ singularity exec test.sif /bin/sh -c 'echo \"hello world\"'
"hello world"

$ singularity run test.sif /bin/sh -c 'echo \"hello world\"'
"hello world"
# no output from example 2.

Now, in the first command, we use exec, which passes all the arguments to /bin/sh in the container, so the output is as we expect.

In the second command, we use run, which will pass all the arguments to the %runscript. So its up to the runscript on what the output is. And if all your %runscript is "$@", then the outputs are identical.

Hopefully that all makes sense, and @ikaneshiro can correct me if I'm wrong.

[WestleyK]

Closing. Reopen if theres any questions.

[mr-c]

Hello @WestleyK , non-members do not have the ability to re-open issues here.

I'm looking for parity with docker run which does respect the quotes

 $ docker run debian /bin/sh -c     'echo \"moo\"'
"moo"

[jmstover]

@mr-c

In the docker://debian image, at the end of the /.singularity.d/runscript file we have:

eval "set ${SINGULARITY_OCI_RUN}"
exec "$@"

The eval call is to expand any variables prior to the exec call. This has the side effect of parsing the command line, which removes a level of quoting. So, taking shell quoting into effect, to replicate what's being done you need to quote one more time.

This way ... after the eval, it leaves it as: echo \"moo\", which gives us:

$ singularity run docker://debian /bin/sh -c 'echo \\\"moo\\\"'
"moo"

Edit: For the exec action, all we do is: exec "$@"

This means what's given on the CLI, gets passed directly to the exec command without another layer of parsing before hand.

You can also test this behavior outside of singularity with:

$ VAR='echo \\\"moo\\\"'
$ eval "set ${VAR}"
$ echo $@
echo "moo"
$ $@
"moo"

In a SIF image, you can replicate it with a small definition file like:

$ cat quick_exec.def
Bootstrap: docker
From: debian

%runscript
exec "$@"

Then doing: singularity run as you are will echo it with quotes.

[mr-c]

Thank you @jmstover ; the context of this question is for running singularity from within cwltool

I'm not sure what you're suggesting that we do here :-/

I'd like to use the singularity run invocation form as we do with docker so we can support users who depend on containers with a default command (ENTRYPOINT) (though I personally consider that an anti-pattern) but that gets us into the quoting problem that started this issue

If we stick with singularity exec then the quoting is correct but we can't support usage of ENTRYPOINT based docker containers.

When in singularity mode we only use Docker containers converted by Singularity itself. So suggestions about writing or editing the SIF image don't appear to be applicable for us.

Any advice as to what we should do here?

(Can someone re-open this issue while we discussion possible solutions? Thanks!)

[jmstover]

I'm not sure there is anything you can do.

When you do a run, there's an extra eval step that happens from the default runscript that gets generated. That parses the command line once to expand any possible variables, and it also ends up removing a level of quoting.

When you do an exec, we take what's given on the CLI, and pass that directly to be executed. No other parsing happens before hand.

Here's the commit that added in that change: https://github.com/sylabs/singularity/commit/61d3fdd8937db809360098e8737a3b1284cf87f3

I'm trying to find out what the exact reasoning for it was.

[AdamSimpson]

I have run into this same issue and it's not easily worked around for our use case. A fix would be much appreciated.

[carterpeel]

Hello,

This is a templated response that is being sent out to all open issues. We are working hard on 'rebuilding' the Singularity community, and a major task on the agenda is finding out what issues are still outstanding.

Please consider the following:

1. Is this issue a duplicate, or has it been fixed/implemented since being added?
2. Is the issue still relevant to the current state of Singularity's functionality?
3. Would you like to continue discussing this issue or feature request?

Thanks, Carter

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in over 60 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[mr-c]

I just confirmed that this issue persists in Singularity 3.8.0

[pedroalvesbatista]

@mr-c We're looking into the issue carefully, soon will bring to community and discuss ways to better solve as well address this. Thankyou for keeping the interest in the subject.

[kmuriki]

Transferred this issue into the new Apptainer repo https://github.com/apptainer/apptainer/issues/1161