Simplify deployment - provide Docker container?

[Masterxilo]

Me and a colleague had some troubles installing and getting this to work, going through the classic dependency hell. The VirtualBox image functions nicely, but we cannot use it in our cluster setup: we cannot run 64 bit virtualbox machines nested inside our private-cloud virtual machines it seems - nested VT-x appears to be hard to do.

The setup instructions on https://github.com/ChrisHIV/shiver/blob/master/info/InstallationNotes.sh where very helpful for installing shiver from scratch. However, they where incomplete in our environment, possibly due to some expected tools not being preinstalled. Also, I don't see any reason not to set up trimmomatic as a system-wide program on the PATH like all the other programs, so the script should just do (2) (or something similar) for us instead of bothering us with a choice for that one (it doesn't with all the other programs). I was able to successfully set up shiver on a bare Ubuntu 16.04 64bit virtual machine using the following steps:

sudo apt update
sudo apt install python-pip -y
pip install --upgrade pip
sudo apt install python3-pip -y
sudo apt install git -y
sudo apt install wget -y
sudo apt install unzip -y
sudo apt install bc -y
cd ~

python --version
python2 --version
python3 --version
bc --version
git --version
unzip --hh
pip --version
pip2 --version
pip3 --version
wget --version

# fastaq
pip3 install pyfastaq

# biopython (import Bio - fix ImportError: No module named Bio)
#sudo pip install biopython
sudo pip2 install biopython
sudo pip3 install biopython


# blast
wget ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.7.1+-x64-linux.tar.gz
tar -xzf ncbi-blast-2.7.1+-x64-linux.tar.gz
echo 'PATH=$PATH:~/ncbi-blast-2.7.1+/bin/' >> ~/.bashrc; source ~/.bashrc
blastx -h

# samtools
# 
# dependency ncurses
sudo apt install -y libncurses5-dev

# 
sudo apt-get install -y zlib1g-dev libbz2-dev liblzma-dev
wget https://github.com/samtools/samtools/releases/download/1.6/samtools-1.6.tar.bz2
tar -xjf samtools-1.6.tar.bz2 
cd ~/samtools-1.6/
./configure
make
sudo make install
echo 'PATH=$PATH:~/samtools-1.6/' >> ~/.bashrc; source ~/.bashrc
cd ~

samtools --version


# mafft
wget https://mafft.cbrc.jp/alignment/software/mafft-7.313-without-extensions-src.tgz
tar -xzf mafft-7.313-without-extensions-src.tgz
cd mafft-7.313-without-extensions/core/
make clean
make
sudo make install
cd ~

mafft --version


# trimmomatic... needs more configuration to be used
sudo apt-get install -y default-jre
wget http://www.usadellab.org/cms/uploads/supplementary/Trimmomatic/Trimmomatic-0.36.zip
unzip Trimmomatic-0.36.zip

#echo "alias trimmomatic='java -jar $HOME/Trimmomatic-0.36/trimmomatic-0.36.jar'" >> ~/.bashrc; source ~/.bashrc
# Aliases are not available outside of bash, need:
mkdir -p ~/bin
echo '#!/bin/bash' >> ~/bin/trimmomatic
echo "java -jar $HOME/Trimmomatic-0.36/trimmomatic-0.36.jar" '"$@"' >> ~/bin/trimmomatic
chmod a=rwx ~/bin/*

echo trimmomatic -version
trimmomatic -version

cd ~

# smalt
wget https://sourceforge.net/projects/smalt/files/latest/download -O smalt.tgz
tar -xzf smalt.tgz
cd smalt-0.7.6/
./configure
make
sudo make install
cd ~

smalt help

# iva (https://github.com/sanger-pathogens/iva http://sanger-pathogens.github.io/iva/)
# its dependencies (that are missing up to this point)
sudo apt install kmc -y
sudo apt install mummer -y
pip3 install iva
cd ~

kmc
#kmc_dump --help
iva --version
nucmer --version

# BWA
git clone https://github.com/lh3/bwa.git
cd bwa
make
echo 'PATH=$PATH:~/bwa/' >> ~/.bashrc; source ~/.bashrc
cd ~

#bwa --version # does not exist

# bowtie2
# isn't that the easiest possible software installation?
wget https://sourceforge.net/projects/bowtie-bio/files/bowtie2/2.3.3.1/bowtie2-2.3.3.1-linux-x86_64.zip/download -O bowtie2.zip 
unzip bowtie2.zip
echo 'PATH=$PATH:~/bowtie2-2.3.3.1-linux-x86_64/' >> ~/.bashrc; source ~/.bashrc
cd ~

bowtie2 --version

# shiver
git clone https://github.com/ChrisHIV/shiver

This has worked nicely on our private cloud setup's Ubuntu 16.04 image and on the Ubuntu 16.04 image from https://www.osboxes.org/ubuntu/ and even inside a docker container starting from the ubuntu:xenial image.

In order to save future researchers from having to figure out this mundane stuff and let them focus on doing research instead, it would be nice if you could give us a more self-contained software package. It would be cool if you guys could provide a more lightweight Docker container which can easily run within another virtualization environment in addition to the VM. The above steps should provide a basis for creating a corresponding Dockerfile.

[ChrisHIV]

Thanks very much for the more complete installation instructions! I'm afraid I have neither experience with Docker nor the time available to do this in the short term, though it would clearly be a plus. To be revisited at some point in the future.

[ChrisHIV]

shiver will henceforth be kept up to date in bioconda, simplifying installation considerably. This also results in automatic generation of docker containers.