New module: Metamaps

[henningonsbring]

Adding MetaMaps, a tool for taxonomic analysis of long-read (PacBio/Oxford Nanopore) metagenomic datasets.

Metamaps database in this PR: https://github.com/nf-core/test-datasets/pull/474/ (merged) Test data to use as input in this PR: https://github.com/nf-core/test-datasets/pull/492 (merged)

PR checklist

• [x] This comment contains a description of changes (with reason).
• [x] If you've fixed a bug or added code that should be tested, add tests!
• [x] If you've added a new tool - have you followed the module conventions in the contribution docs
• [x] If necessary, include test data in your PR.
• [x] Remove all TODO statements.
• [x] Emit the versions.yml file.
• [x] Follow the naming conventions.
• [x] Follow the parameters requirements.
• [x] Follow the input/output options guidelines.
• [ ] Add a resource label
• [ ] Use BioConda and BioContainers if possible to fulfil software requirements.
• Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
  • [ ] PROFILE=docker pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
  • [ ] PROFILE=singularity pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
  • [ ] PROFILE=conda pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware

[henningonsbring]

This is basically ready for review, I would like to be able to run the command: nf-core modules create-test-yml metamaps/mapDirectly

But when running that command I get the following error: No such file: https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/sarscov2/nanopore/fastq/test_2.fastq.gz

I guess that is because I need this PR merged: https://github.com/nf-core/test-datasets/pull/492

[sateeshperi]

ok lets wait for https://github.com/nf-core/test-datasets/pull/492 to be merged now

[henningonsbring]

ok lets wait for nf-core/test-datasets#492 to be merged now

Let's wait for that yes, thanks a lot for the comments so far 👍

[sateeshperi]

you might have to test locally first after the recent merge

[henningonsbring]

you might have to test locally first after the recent merge

Yes, I assume there are a few errors to fix. When I run nf-core modules create-test-yml metamaps/mapdirectly then I get: MarkupError: closing tag '[/nf-core/test-datasets/modules/data/genomics/sarscov2/genome/db/metamaps.tar.gz]' at position 595 doesn't match any open tag

[henningonsbring]

Current error me and @sofstam are trying to fix: When the test for metamaps classify is invoked then we get the following error:

         Command error:                                                                                                                                                                                                                                                                          
           terminate called after throwing an instance of 'boost::wrapexcept<std::logic_error>'                                                                                                                                                                                                  
             what():  Attempt to access an uninitialized boost::match_results<> class.                                                                                                                                                                                                           
           .command.sh: line 7:     8 Aborted                 metamaps classify --mappings test.classification_res --threads 2 --DB metamaps_db                                                                                                                                                  
                                                                                                                                                                                                                                                                                                 
         Work dir:                                                                                                                                                                                                                                                                               
           /var/folders/s6/m7vtlhl147bfm538q4cb_z694thzg_/T/tmpbit2j5d0/7b/97b74dbb66cc436f08b10fd0e894ad 

The reason why we get confused by this error is because in the Work dir we can see that all the files are present that should allow us to run metamaps classify. Further if we change the directory to the Work dir via cd /var/folders/s6/m7vtlhl147bfm538q4cb_z694thzg_/T/tmpbit2j5d0/7b/97b74dbb66cc436f08b10fd0e894ad and then print the content of the command-file (.command.sh) we see the exact command that we want to run:

metamaps \
    classify \
     \
    --mappings test.classification_res \
    --threads 2 \
    --DB metamaps_db

Further if I then invoke the command file with bash .command.sh then metamaps classify works, given the output:

bash .command.sh
>>>>>>>>>>>>>>>>>>
DB = metamaps_db
Mappings = test.classification_res
Min. reads for 'U' = 10000
Threads = 2
>>>>>>>>>>>>>>>>>>
Read taxonomy from metamaps_db/taxonomy -- have 14 nodes.
Starting EM...
EM round 0

        Log likelihood: -4066.04
EM round 1

        Log likelihood: -4066.04
        Improvement: 0
        Relative   : 1
Outputting mappings with adjusted alignment qualities.

Since running .command.sh leads to a successfully metamaps run I assume that there is an error in metamaps or Docker.

[sateeshperi]

from your local run log Read taxonomy from metamaps_db/taxonomy looks like your process might need /taxonomy added to --DB metamaps_db - > --DB metamaps_db/taxonomy

just a thought

[henningonsbring]

from your local run log Read taxonomy from metamaps_db/taxonomy looks like your process might need /taxonomy added to --DB metamaps_db - > --DB metamaps_db/taxonomy

just a thought

I do not think this is the case. I can change directory to the workdir, there I find the unziped metamaps database in the folder metamaps_db. From that folder I then run metamaps classify --mappings test.classification_res --DB metamaps_db, and get the output that confirms everything is working:

>>>>>>>>>>>>>>>>>>
DB = metamaps_db
Mappings = test.classification_res
Min. reads for 'U' = 10000
Threads = 1
>>>>>>>>>>>>>>>>>>
Read taxonomy from metamaps_db/taxonomy -- have 14 nodes.
Starting EM...
EM round 0

        Log likelihood: -4066.04
EM round 1

        Log likelihood: -4066.04
        Improvement: 0
        Relative   : 1
Outputting mappings with adjusted alignment qualities.

for the --DB flag you are supposed to give the root folder to the database, not the taxonomy folder inside the database folder.

[maxulysse]

the metamaps/classify tests are all failing, did you manage to make them work locally?

[henningonsbring]

the metamaps/classify tests are all failing, did you manage to make them work locally?

No, the tests fail both locally and on the HPC I got access to. The problem associated with the classify-test is described above in the comment starting with "Current error me and @sofstam are trying to fix:". We have contacted the developer of metamaps.

[matthdsm]

Hi there!

We've noticed there hasn't been much activity here. Are you still planning on working on this? If not, you can ignore this message and we'll close your PR in about 2 weeks

Cheers the nf-core maintainers

[sofstam]

New bioconda recipe that seems to work so far so I am reopening this.