make singularity container

[colindaven]

TODO

• [ ] https://github.com/grst/containerize-conda New method, conda-pack, looks great! https://conda.github.io/conda-pack/
• [ ] test conda in container --- > see https://stackoverflow.com/questions/54678805/containerize-a-conda-environment-in-a-singularity-container
• [x] make on basis Ubuntu20.04
• [ ] problems: - sourcing env variables for script from singularity is very difficult (sh, not bash, no source command). Even more tricky as yaml is user defined
• [ ] test usage in slurm
• [ ] document - singularity tutorial
• [ ] mamba container as base ! Use mamba container..(but can it read a conda env file ?) https://www.biostars.org/p/9476786/
• [x] added config.yml to make Wochenende config options independent of invidual scripts

Advantages : more reproducible. Not dependent on changing conda libs. Distributable as single file Disadvantages: requires singularity install. More technical to change reference FASTA paths.

[colindaven]

continuumio build on basis of debian, not ubuntu.

Need to set shell to bash properly

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run

    $ conda init <SHELL_NAME>

Currently supported shells are:
  - bash
  - fish
  - tcsh
  - xonsh
  - zsh
  - powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

[colindaven]

More progress made in September

Outstanding questions:

• [ ] modification of SLURM config? Not really possible in read only container (use srun, not sbatch)
• [ ] edit - link file config.yaml from current dir outside container to container dir /data/wochenende (to allow modifying refseqs)
• [ ] test
• [ ] Workaround: only direct use of python script with full parameters ?
• [ ] call container from runbatch srun script with full params ? <-- most realistic
• [ ] writeable container where users change config.yml in container?

[colindaven]

@B1T0 The first item in the list , https://github.com/grst/containerize-conda , could be the most promising. Haven't tried it yet, but much simpler than the others.

[B1T0]

I will have a look at this now

[colindaven]

Use mamba container to make conda in container easier ? (but can it read a conda env file ?)

https://www.biostars.org/p/9476786/

[colindaven]

It looks really difficult / impossible to use the bash specific setup.sh and current bash methods for reading in the config.yaml because singularity does not have a good/easy source command.

It will be even more difficult to allow users to edit the config.yaml and have it sourced at runtime ...