/etc/mtab doesn't always exist

[jmstover]

Some container images do not contain the /etc/mtab link. For example:

$ singularity exec docker://ubuntu:latest ls -ld /etc/mtab
ls: cannot access '/etc/mtab': No such file or directory
$ singularity exec docker://debian:latest ls -ld /etc/mtab
ls: cannot access '/etc/mtab': No such file or directory

Compare this to:

$ singularity exec docker://centos:latest ls -ld /etc/mtab
lrwxrwxr-x 1 root root 19 Aug 12 19:00 /etc/mtab -> ../proc/self/mounts

During image build, we should check for the existence of /etc/mtab and if it does not exist, create a symlink to /proc/self/mounts in the container image.

[boegel]

I think I'm hitting this when starting a Docker container with Singularity while using --fusemount "container:fuse-overlayfs ...":

fusermount: failed to open /etc/mtab: No such file or directory
fusermount: mountpoint /proc/62/fd/3 is not a directory or a regular file
failed to create Fuse channel

I tried working around this (perhaps naively) by creating the /etc/mtab symlink myself when building the Docker image, but then I hit this:

ln: failed to create symbolic link '/etc/mtab': File exists

So /etc/mtab is there in the Docker image, but Singularity isn't picking up on it when ingesting the Docker container image?

Is there a known workaround for this, either in the Docker image, or when using Singularity?

[dtrudg]

@boegel - what's the docker image and the full command here?

fusermount: mountpoint /proc/62/fd/3 is not a directory or a regular file

is generally indicative of an older version of fuse that doesn't support mounting from a file descriptor... and maybe that is the hard failure rather than the mtab issue.

Per the guide you need libfuse3 - https://sylabs.io/guides/3.6/user-guide/bind_paths_and_mounts.html?highlight=fuse#fuse-mounts

Can you confirm what you are using?

[carterpeel]

Hello,

This is a templated response that is being sent out to all open issues. We are working hard on 'rebuilding' the Singularity community, and a major task on the agenda is finding out what issues are still outstanding.

Please consider the following:

1. Is this issue a duplicate, or has it been fixed/implemented since being added?
2. Is the issue still relevant to the current state of Singularity's functionality?
3. Would you like to continue discussing this issue or feature request?

Thanks, Carter

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in over 60 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[pedroalvesbatista]

@jmstover Still getting this problem ? Have this been solved already ? If yes, what work around have you followed and applied ?

We're looking into the issue carefully, soon will bring to community and discuss ways to better solve as well address this. Thankyou for keeping interest over the subject.

[kmuriki]

Sounds like an issue with the older version of fuse that doesn't support mounting from a file descriptor. Not sure if this is still relevant, so closing this old, stale issue.

[bennahugo]

Still an issue with apptainer 1.2.3 (apptainer-suid from your Ubuntu PPA - edit: on Kubuntu 20.04 LTS)

I am building a package with Docker (tagged tart here) and then running

sudo singularity build tart.sif docker-daemon://tart:latest 

to import the image into apptainer. Running

> $ apptainer exec tart.sif /bin/bash                                                                                                                                                                                                                                         
INFO:    /etc/singularity/ exists; cleanup by system administrator is not complete (see https://apptainer.org/docs/admin/latest/singularity_migration.html)
Apptainer> ls /etc/mtab
ls: cannot access '/etc/mtab': No such file or directory

shows it does not exist. Inside the software FUSE subsequently fails (libfuse2 and fuse3 are both installed)

fusermount: failed to open /etc/mtab: No such file or directory
fusermount: mount failed: Operation not permitted
open dir error: No such file or directory

If I run it via Docker /etc/mtab is available

docker run -it --rm --entrypoint /bin/bash tart                                                                                                                                                                                                                           
root@5848d30cfb1c:/# ls /etc/mtab 
/etc/mtab

[DrDaveD]

If you still have a problem please open a new apptainer ticket.