nf-core
New module: checkqc
Adding new module checkQC. Related to issue: https://github.com/nf-core/modules/issues/4011
Depends on https://github.com/nf-core/test-datasets/pull/1024 for tests to work.
Note: The CheckQC version used in the module is not the latest. I was forced to use an old one since there is an issue with the biocontainers for more recent ones. I aim to fix this and make a PR to upgrade to that version later.
PR checklist
Closes #XXX
- [X] This comment contains a description of changes (with reason).
- [X] If you've fixed a bug or added code that should be tested, add tests!
- [X] If you've added a new tool - have you followed the module conventions in the contribution docs
- [X] Remove all TODO statements.
- [X] Emit the
versions.ymlfile. - [X] Follow the naming conventions.
- [X] Follow the parameters requirements.
- [X] Follow the input/output options guidelines.
- [X] Add a resource
label - [X] Use BioConda and BioContainers if possible to fulfil software requirements.
- Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
- [ ]
PROFILE=docker pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware - [ ]
PROFILE=singularity pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware - [ ]
PROFILE=conda pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
- [ ]
could you work on the failing tests and reach us out on nf-core/nf-test slack channel if you need help. cheers
Yes, would also prefer to just have one config called nextflow.config. Not sure how to have it once with extra arga and once without
Yeah, exactly, I'm not super happy about the two configs, but it was a convenient way to pass different ext.args. If there's another way to do that I'm happy to change it.
Also, I now noticed that my tests are failing. I can re-create it with Docker, but not singularity. Will investigate further.
I'm surprised the conda and singularity tests still are failing, can't reproduce that locally. I do still have issues with the docker image locally, made an issue in bioconda-recipes about it :point_up:
Will address the lint issues.
Hi @matrulda are you planning on finishing this PR? I need this module soon so I can finish it for you if you don't have the time for it :)
@nvnieuwk Thank you! I've put it on hold since the biocontainers for CheckQC aren't really working at the moment, see https://github.com/bioconda/bioconda-recipes/issues/46558 . Getting help with it here: https://github.com/bioconda/bioconda-recipes/pull/40431. Feel very free to work on solving this issue as well as making changes to this PR. I appreciate any help I can get :)
Hey @matrulda and @nvnieuwk, I've written a new bioconda recipe that could solve the biocontainers issue by using the latest 4.0.4 checkqc. Could you please take a look and review it if you can? Thank you :)
Thanks for replying @nvnieuwk but I meant a review for the bioconda-recipes submission. I have not made any modifications to this module yet :) We can discuss it next week, though.
@fmalmeida adding the environment.yml with the version results in linting failure. This is related to this issue.
The environment.yml of the module checkqc is not valid: {'pip': ['checkqc==3.8.0', 'interop==1.2.4']} is not of type 'string'. Check the entry for dependencies.
This was observed in this module as well.
In addition, even when removing versions, the conda tests do not pass due to libmamba being used in the test.yml:
│ Test [a030344b] 'homo_sapiens illumina test_interop_bcl2fastqstats_flowcell - QC OK' FAILED (16.366s) │
│ │
│ Assertion failed: │
│ │
│ 2 of 2 assertions failed │
│ │
│ Nextflow stdout: │
│ │
│ ERROR ~ Error executing process > 'UNTAR (1)' │
│ │
│ Caused by: │
│ Failed to create Conda environment │
│ command: conda env create --prefix │
│ /workspace/modules/.nf-test/tests/a030344b93d4d35baf2381e4cf0f249f/work/conda/untar-6ec82032d38a28f610b86307ce4a0ff3 --file │
│ /workspace/modules/./modules/nf-core/checkqc/tests/../../untar/environment.yml │
│ status : 1 │
│ message: │
│ Error while loading conda entry point: conda-libmamba-solver (libarchive.so.19: cannot open shared object file: No such file or directory) │
│ Retrieving notices: ...working... done │
│ │
│ CondaValueError: You have chosen a non-default solver backend (libmamba) but it was not recognized. Choose one of: classic │
@fmalmeida adding the
environment.ymlwith the version results in linting failure. This is related to this issue.The environment.yml of the module checkqc is not valid: {'pip': ['checkqc==3.8.0', 'interop==1.2.4']} is not of type 'string'. Check the entry for dependencies.This was observed in this module as well.
In addition, even when removing versions, the conda tests do not pass due to
libmambabeing used in thetest.yml:│ Test [a030344b] 'homo_sapiens illumina test_interop_bcl2fastqstats_flowcell - QC OK' FAILED (16.366s) │ │ │ │ Assertion failed: │ │ │ │ 2 of 2 assertions failed │ │ │ │ Nextflow stdout: │ │ │ │ ERROR ~ Error executing process > 'UNTAR (1)' │ │ │ │ Caused by: │ │ Failed to create Conda environment │ │ command: conda env create --prefix │ │ /workspace/modules/.nf-test/tests/a030344b93d4d35baf2381e4cf0f249f/work/conda/untar-6ec82032d38a28f610b86307ce4a0ff3 --file │ │ /workspace/modules/./modules/nf-core/checkqc/tests/../../untar/environment.yml │ │ status : 1 │ │ message: │ │ Error while loading conda entry point: conda-libmamba-solver (libarchive.so.19: cannot open shared object file: No such file or directory) │ │ Retrieving notices: ...working... done │ │ │ │ CondaValueError: You have chosen a non-default solver backend (libmamba) but it was not recognized. Choose one of: classic │
Hi @atrigila ,
I see. Then unfortunately I think the best is to keep the env.yml file using the bioconda installation, however, using the same version that is used in the container. And then, just remove the conda testing profile from the CI, since the container executions work.
@fmalmeida I think I prefer to stick to the current version as it specifically states that it is installed from pip. Since there is at least other module that has this same linting issue, and I think this will be addressed in the future, I would ask core-members @edmundmiller and @FloWuenne:
- Can you disable the
condaCI .yml for checkqc? As we do not have permissions to do so. - Could you allow us to merge with the linting issue (the same observed in module spotiflow? Thank you!
conda CI is properly disabled. It was actually disabled for pytest, and it should have been disabled for nf-test actually.
