drop endedness check for fastqc, add tests

[matthdsm]

Fixes #2278

PR checklist

Closes #XXX

• [ ] This comment contains a description of changes (with reason).
• [ ] If you've fixed a bug or added code that should be tested, add tests!
• [ ] If you've added a new tool - have you followed the module conventions in the contribution docs
• [ ] If necessary, include test data in your PR.
• [ ] Remove all TODO statements.
• [ ] Emit the versions.yml file.
• [ ] Follow the naming conventions.
• [ ] Follow the parameters requirements.
• [ ] Follow the input/output options guidelines.
• [ ] Add a resource label
• [ ] Use BioConda and BioContainers if possible to fulfil software requirements.
• Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
  • [ ] PROFILE=docker pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
  • [ ] PROFILE=singularity pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
  • [ ] PROFILE=conda pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware

[matthdsm]

Note, this does NOT fix bam inputs, which will need some more edits

[matthdsm]

@nf-core-bot fix linting

[mahesh-panchal]

Does the code still work if the single file is not a list? I.e. the channel structure is [ meta, file ] instead of [ meta, [ file ] ]

[matthdsm]

Looks OK to me, what do you think abotu adding a little check that if someone gives more than 2 files it errors out?

(I'm thinking in cases where you have R1, R2, + orphans of pairs reads as three different input files which the module does not support).

If we want to support more than 2 files I guess the best course to go is to drop all the symlink magic

[mahesh-panchal]

Doesn't look there's a need for the if if both code blocks are the same.

[matthdsm]

Doesn't look there's a need for the if if both code blocks are the same.

GREAT point 😅

[mahesh-panchal]

Just to note now that $prefix is also now redundant. Should there be an effort to do the renaming so $prefix is not redundant?

[matthdsm]

Just to note now that $prefix is also now redundant. Should there be an effort to do the renaming so $prefix is not redundant?

Yes, I think that was mainly what all the symlinking magic was about. The issue with the current implementation was that is was quite limiting. I'd have to think about how I (we?) should handle multiple inputs and input types.

[matthdsm]

How did a multiqc test get in there??

[matthdsm]

@mahesh-panchal , feel free to commit directly to this PR if you feel like it.

[matthdsm]

Seems like we'll have to update the test.yml for upstream subworkflows too

[mahesh-panchal]

What broke? The file name or something else?

[matthdsm]

Seems like the names for the output files changed

FAILED tests/subworkflows/nf-core/fastq_fastqc_umitools_trimgalore/test.yml::fastq_fastqc_umitools_trimgalore test_fastq_fastqc_umitools_trimgalore_single::output/fastqc/test_fastqc.html::should exist
[116](https://github.com/nf-core/modules/actions/runs/3234898533/jobs/5298593145#step:11:117)
FAILED tests/subworkflows/nf-core/fastq_fastqc_umitools_trimgalore/test.yml::fastq_fastqc_umitools_trimgalore test_fastq_fastqc_umitools_trimgalore_single::output/fastqc/test_fastqc.html::md5sum
[117](https://github.com/nf-core/modules/actions/runs/3234898533/jobs/5298593145#step:11:118)
FAILED tests/subworkflows/nf-core/fastq_fastqc_umitools_trimgalore/test.yml::fastq_fastqc_umitools_trimgalore test_fastq_fastqc_umitools_trimgalore_single::output/fastqc/test_fastqc.zip::should exist

[matthdsm]

I'm not sure what's going wrong here, when I rerun create-test-yml there's no output...

INFO     Building test meta for entry point 'test_fastq_fastqc_umitools_trimgalore_skip'                               test_yml_builder.py:163
INFO     Running 'fastq_fastqc_umitools_trimgalore' test with command:                                                 test_yml_builder.py:348
         nextflow run ./tests/subworkflows/nf-core/fastq_fastqc_umitools_trimgalore -entry                                                    
         test_fastq_fastqc_umitools_trimgalore_skip -c ./tests/config/nextflow.config --outdir /tmp/tmpmga0mpao                               
         -work-dir /tmp/tmphx216gx3                                                                                                           
Picked up JAVA_TOOL_OPTIONS:  -Xmx3489m
INFO     Repeating test ...                                                                                            test_yml_builder.py:351
Picked up JAVA_TOOL_OPTIONS:  -Xmx3489m
INFO     Test workflow finished!                                                                                       test_yml_builder.py:365
CRITICAL Could not find any test result files in '/tmp/tmpmga0mpao'  

[matthdsm]

Ready to go I think!

[mahesh-panchal]

Hmm. Why are the fastq's different in the failing tests? This shouldn't be something that should differ between conda and containers.

[matthdsm]

I have no idea... There must've been some change somewhere, I've been seeing wonky conda checksums all morning

[mahesh-panchal]

I dug into it and got this far:

• Text content between two files is the same.
• There is a slight compression difference between conda environment and docker environment.

$ gzip -l /tmp/pytest_workflow_2owhoa31/fastq_fastqc_umitools_trimgalore_test_fastq_fastqc_umitools_trimgalore_single/output/trimgalore/test_trimmed.fq.gz /tmp/pytest_workflow_kxh7e0mo/fastq_fastqc_umitools_trimgalore_test_fastq_fastqc_umitools_trimgalore_single/output/trimgalore/test_trimmed.fq.gz
         compressed        uncompressed  ratio uncompressed_name
               9489               34198  72.3% /tmp/pytest_workflow_2owhoa31/fastq_fastqc_umitools_trimgalore_test_fastq_fastqc_umitools_trimgalore_single/output/trimgalore/test_trimmed.fq
               9485               34198  72.3% /tmp/pytest_workflow_kxh7e0mo/fastq_fastqc_umitools_trimgalore_test_fastq_fastqc_umitools_trimgalore_single/output/trimgalore/test_trimmed.fq
              18974               68396  72.3% (totals)

[matthdsm]

Thank you for looking into it!