modules icon indicating copy to clipboard operation
modules copied to clipboard

`PURGEDUPS_PURGEDUPS` migration to nf-test

Open TomHarrop opened this issue 9 months ago • 4 comments

PR checklist

Closes #7674

  • [ ] This comment contains a description of changes (with reason).
  • [ ] If you've fixed a bug or added code that should be tested, add tests!
  • [ ] If you've added a new tool - have you followed the module conventions in the contribution docs
  • [ ] If necessary, include test data in your PR.
  • [ ] Remove all TODO statements.
  • [ ] Emit the versions.yml file.
  • [ ] Follow the naming conventions.
  • [ ] Follow the parameters requirements.
  • [ ] Follow the input/output options guidelines.
  • [ ] Add a resource label
  • [ ] Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • [ ] nf-core modules test <MODULE> --profile docker
      • [ ] nf-core modules test <MODULE> --profile singularity
      • [ ] nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • [ ] nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • [ ] nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • [ ] nf-core subworkflows test <SUBWORKFLOW> --profile conda

TomHarrop avatar Mar 24 '25 04:03 TomHarrop

Thanks, I think those are done.

TomHarrop avatar Mar 25 '25 02:03 TomHarrop

@SPPearce looks like that issue using --stub is discussed here: https://github.com/nextflow-io/nextflow/issues/5456

 Cannot invoke method clone() on null object

The conda test segfaults, not sure if this matters as both the container tests pass.

    > Command error:
    >   .command.sh: line 2:  1960 Segmentation fault      (core dumped) calcuts test.PB.stat > test.cutoffs 2> >(tee test.calcuts.log >&2)

TomHarrop avatar Mar 25 '25 05:03 TomHarrop

That generally means you need to add stubs to the other modules you are using to setup the tests, can you try doing that in this PR please

SPPearce avatar Mar 25 '25 05:03 SPPearce

Oh great, thanks for the tip. It's just the failing conda tests now.

TomHarrop avatar Mar 25 '25 06:03 TomHarrop

Thanks @SPPearce , that should be fixed now. It was an issue with the way the input had been specified for pytest.

TomHarrop avatar Mar 26 '25 02:03 TomHarrop

CALCUTS conda tests are failing with a segmentation fault, but separately the other module conda tests are failing because of a discrepancy in the versions.yml.

It looks like the conda version should be the same as that in the biocontainers images but I can't test it locally to see what the actual output and difference is

awgymer avatar Mar 26 '25 02:03 awgymer

The version is being returned as 1.25 or 1.26 between the different container methods

SPPearce avatar Apr 10 '25 06:04 SPPearce

Yeah conda returns 1.2.6 while docker returns 1.2.5

famosab avatar Apr 10 '25 07:04 famosab

Lets swap to seqera containers I would suggest?

Docker: community.wave.seqera.io/library/purge_dups:1.2.6--1966ab26985f9f67 Singularity: https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/8a/8afcc1222a0512a4fae7a7e7315ccfc841f3578df600b99d3dc563c3a8361352/data

famosab avatar May 20 '25 08:05 famosab

@TomHarrop I do not have write access to your fork but this is what you need to replace:

    container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
        'https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/8a/8afcc1222a0512a4fae7a7e7315ccfc841f3578df600b99d3dc563c3a8361352/data':
        'community.wave.seqera.io/library/purge_dups:1.2.6--1966ab26985f9f67' }"

famosab avatar May 20 '25 08:05 famosab

@famosab if you want to suggest those changes I will accept them - that way your commit is tracked.

Otherwise LMK if you want me to just make the change.

Thanks!

TomHarrop avatar May 21 '25 08:05 TomHarrop

@famosab if you want to suggest those changes I will accept them - that way your commit is tracked.

Otherwise LMK if you want me to just make the change.

Thanks!

Just make the change yourself please, I'm sure Famke won't mind it isn't in her name on the commit.

SPPearce avatar May 21 '25 08:05 SPPearce

OK done, hope that was the file you were referring to.

TomHarrop avatar May 21 '25 08:05 TomHarrop

Could you also re-run nf-test? :) And update the snaps?

famosab avatar May 21 '25 08:05 famosab

Also you should remove the entry in the tests/config/pytest_modules.yml file to stop the pytest from running.

SPPearce avatar May 21 '25 08:05 SPPearce

Also you should remove the entry in the tests/config/pytest_modules.yml file to stop the pytest from running.

I think I did, just not for the other submodules.

TomHarrop avatar May 21 '25 09:05 TomHarrop

There is https://github.com/nf-core/modules/pull/8472 and https://github.com/nf-core/modules/pull/8471 which might need to be merged first for this to work :)

famosab avatar May 21 '25 11:05 famosab