tools icon indicating copy to clipboard operation
tools copied to clipboard

Include singularity in docker image, for downloading pipelines

Open fa2k opened this issue 2 years ago • 3 comments

Is your feature request related to a problem? Please describe

The docker image doesn't contain singularity. I'd like to use the "download" command to download pipelines with singularity images.

If I try to download a pipeline including images, using the docker image, I get this message:

CRITICAL Singularity is needed to pull images, but it is not installed                                                                                                download.py:182

Describe the solution you'd like

Maybe singularity could be included. But it's a pain to build on Debian/Ubuntu, and would triple the size of the Dockerfile (source code).

Describe alternatives you've considered

I've created an alternative dockerfile to use for this. Edit: It doesn't work. It doesn't write the singularity images.

Additional context

tools-singularity.Dockerfile (doesn't work)

FROM centos:8

RUN yum install -y epel-release
RUN yum install -y java-11-openjdk singularity git python38
RUN alternatives --set python /usr/bin/python3

# Do not pick up python packages from $HOME
ENV PYTHONNUSERSITE=1

# Update pip to latest version
RUN python -m pip install --upgrade pip

# Install dependencies
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt

# Setup ARG for NXF_VER ENV
ARG NXF_VER=""
ENV NXF_VER ${NXF_VER}
# Install Nextflow
RUN curl -s https://get.nextflow.io | bash \
      && mv nextflow /usr/local/bin \
      && chmod a+rx /usr/local/bin/nextflow
# Add the nf-core source files to the image
COPY . /usr/src/nf_core
WORKDIR /usr/src/nf_core

# Install nf-core
RUN python -m pip install .

# Set up entrypoint and cmd for easy docker usage
ENTRYPOINT [ "nf-core" ]
CMD [ "." ]

fa2k avatar Nov 15 '21 15:11 fa2k

You should be able to directly download the images hosted by galaxy, but I agree to get the singularity conversion of the dockerhub images you do need Singularity. Maybe it should be added in the docs, I'm thinking that most people that download for offline usage have (or at least should have) singularity installed on a system that can communicate with their offline server.

maxulysse avatar Dec 08 '21 11:12 maxulysse

Yes it works perfectly for the rnaseq pipeline. Downloads all the images.

Maybe it's not that common, but our system is completely offline. I was hoping to transfer the pipeline to it using an external USB drive. We don't have an Internet-based system with singularity at the moment, but we can probably set that up, and install nf-core with conda. Thanks a lot for reading and considering the issue.

fa2k avatar Dec 08 '21 17:12 fa2k

It is pretty common, this is the main use case that we wrote nf-core download for. But I guess most people run nf-core tools natively rather than inside Docker.

I agree that it would be nice to add in Singularity to make this easier, however it strikes me as the kind of thing that could be very difficult to run well.. (But I haven't tried)

ewels avatar Dec 09 '21 23:12 ewels