sarek
sarek copied to clipboard
Update CNVkit
PR checklist
Overall description
This PR adds the following improvements to the CNVkit subworkflow(s):
- updates the CNVkit modules, as updated in nf-core/modules
- with the revised cnvkit/batch module, makes handling
params.cnvkit_reference
uniform across the three subworkflows (germline, tumour only, tumour-normal) - adds a specific
cnvkit/call
module to perform calling fromcns
files - adds the
cnvkit/export
module, to export called CNVs into VCF, thus uniforming Sarek variant results - adds the recommended settings to
cnvkit/call
config for germline workflows
Checklist
- [ ] The code has been tested on real germline samples, both from FASTQ and BAM files: the automated tests might still need a bit of work
- [ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
- [ ] If necessary, also make a PR on the nf-core/sarek branch on the nf-core/test-datasets repository.
- [ ] Make sure your code lints (
nf-core lint
). - [ ] Ensure the test suite passes (
nf-test test tests/ --verbose --profile +docker
). - [ ] Check for unexpected warnings in debug mode (
nextflow run . -profile debug,test,docker --outdir <OUTDIR>
). - [ ] Usage Documentation in
docs/usage.md
is updated. - [ ] Output Documentation in
docs/output.md
is updated. - [ ]
CHANGELOG.md
is updated. - [ ]
README.md
is updated (including new tool citations and authors/contributors).
nf-core lint
overall result: Passed :white_check_mark: :warning:
Posted for pipeline commit bb0ec8d
+| ✅ 197 tests passed |+
#| ❔ 15 tests were ignored |#
!| ❗ 3 tests had warnings |!
:heavy_exclamation_mark: Test warnings:
-
pipeline_todos - TODO string in
main.nf
: Optionally add in-text citation tools to this list. -
pipeline_todos - TODO string in
main.nf
: Optionally add bibliographic entries to this list. -
pipeline_todos - TODO string in
main.nf
: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
:grey_question: Tests ignored:
-
files_exist - File is ignored:
.github/workflows/awsfulltest.yml
-
files_exist - File is ignored:
.github/workflows/awstest.yml
-
files_exist - File is ignored:
conf/modules.config
-
files_exist - File is ignored:
lib/WorkflowMain.groovy
-
files_exist - File is ignored:
lib/NfcoreTemplate.groovy
-
files_exist - File is ignored:
lib/WorkflowSarek.groovy
-
files_unchanged - File ignored due to lint config:
.github/PULL_REQUEST_TEMPLATE.md
-
files_unchanged - File ignored due to lint config:
assets/nf-core-sarek_logo_light.png
-
files_unchanged - File ignored due to lint config:
docs/images/nf-core-sarek_logo_light.png
-
files_unchanged - File ignored due to lint config:
docs/images/nf-core-sarek_logo_dark.png
-
files_unchanged - File ignored due to lint config:
.gitignore
or.prettierignore
- actions_ci - actions_ci
-
actions_awstest - 'awstest.yml' workflow not found:
/home/runner/work/sarek/sarek/.github/workflows/awstest.yml
- template_strings - template_strings
- modules_config - modules_config
:white_check_mark: Tests passed:
-
files_exist - File found:
.gitattributes
-
files_exist - File found:
.gitignore
-
files_exist - File found:
.nf-core.yml
-
files_exist - File found:
.editorconfig
-
files_exist - File found:
.prettierignore
-
files_exist - File found:
.prettierrc.yml
-
files_exist - File found:
CHANGELOG.md
-
files_exist - File found:
CITATIONS.md
-
files_exist - File found:
CODE_OF_CONDUCT.md
-
files_exist - File found:
LICENSE
orLICENSE.md
orLICENCE
orLICENCE.md
-
files_exist - File found:
nextflow_schema.json
-
files_exist - File found:
nextflow.config
-
files_exist - File found:
README.md
-
files_exist - File found:
.github/.dockstore.yml
-
files_exist - File found:
.github/CONTRIBUTING.md
-
files_exist - File found:
.github/ISSUE_TEMPLATE/bug_report.yml
-
files_exist - File found:
.github/ISSUE_TEMPLATE/config.yml
-
files_exist - File found:
.github/ISSUE_TEMPLATE/feature_request.yml
-
files_exist - File found:
.github/PULL_REQUEST_TEMPLATE.md
-
files_exist - File found:
.github/workflows/branch.yml
-
files_exist - File found:
.github/workflows/ci.yml
-
files_exist - File found:
.github/workflows/linting_comment.yml
-
files_exist - File found:
.github/workflows/linting.yml
-
files_exist - File found:
assets/email_template.html
-
files_exist - File found:
assets/email_template.txt
-
files_exist - File found:
assets/sendmail_template.txt
-
files_exist - File found:
assets/nf-core-sarek_logo_light.png
-
files_exist - File found:
conf/test.config
-
files_exist - File found:
conf/test_full.config
-
files_exist - File found:
docs/images/nf-core-sarek_logo_light.png
-
files_exist - File found:
docs/images/nf-core-sarek_logo_dark.png
-
files_exist - File found:
docs/output.md
-
files_exist - File found:
docs/README.md
-
files_exist - File found:
docs/README.md
-
files_exist - File found:
docs/usage.md
-
files_exist - File found:
main.nf
-
files_exist - File found:
assets/multiqc_config.yml
-
files_exist - File found:
conf/base.config
-
files_exist - File found:
conf/igenomes.config
-
files_exist - File found:
modules.json
-
files_exist - File not found check:
.github/ISSUE_TEMPLATE/bug_report.md
-
files_exist - File not found check:
.github/ISSUE_TEMPLATE/feature_request.md
-
files_exist - File not found check:
.github/workflows/push_dockerhub.yml
-
files_exist - File not found check:
.markdownlint.yml
-
files_exist - File not found check:
.nf-core.yaml
-
files_exist - File not found check:
.yamllint.yml
-
files_exist - File not found check:
bin/markdown_to_html.r
-
files_exist - File not found check:
conf/aws.config
-
files_exist - File not found check:
docs/images/nf-core-sarek_logo.png
-
files_exist - File not found check:
lib/Checks.groovy
-
files_exist - File not found check:
lib/Completion.groovy
-
files_exist - File not found check:
lib/Utils.groovy
-
files_exist - File not found check:
lib/Workflow.groovy
-
files_exist - File not found check:
parameters.settings.json
-
files_exist - File not found check:
pipeline_template.yml
-
files_exist - File not found check:
Singularity
-
files_exist - File not found check:
lib/nfcore_external_java_deps.jar
-
files_exist - File not found check:
.travis.yml
-
nextflow_config - Config variable found:
manifest.name
-
nextflow_config - Config variable found:
manifest.nextflowVersion
-
nextflow_config - Config variable found:
manifest.description
-
nextflow_config - Config variable found:
manifest.version
-
nextflow_config - Config variable found:
manifest.homePage
-
nextflow_config - Config variable found:
timeline.enabled
-
nextflow_config - Config variable found:
trace.enabled
-
nextflow_config - Config variable found:
report.enabled
-
nextflow_config - Config variable found:
dag.enabled
-
nextflow_config - Config variable found:
process.cpus
-
nextflow_config - Config variable found:
process.memory
-
nextflow_config - Config variable found:
process.time
-
nextflow_config - Config variable found:
params.outdir
-
nextflow_config - Config variable found:
params.input
-
nextflow_config - Config variable found:
params.validationShowHiddenParams
-
nextflow_config - Config variable found:
params.validationSchemaIgnoreParams
-
nextflow_config - Config variable found:
manifest.mainScript
-
nextflow_config - Config variable found:
timeline.file
-
nextflow_config - Config variable found:
trace.file
-
nextflow_config - Config variable found:
report.file
-
nextflow_config - Config variable found:
dag.file
-
nextflow_config - Config variable (correctly) not found:
params.nf_required_version
-
nextflow_config - Config variable (correctly) not found:
params.container
-
nextflow_config - Config variable (correctly) not found:
params.singleEnd
-
nextflow_config - Config variable (correctly) not found:
params.igenomesIgnore
-
nextflow_config - Config variable (correctly) not found:
params.name
-
nextflow_config - Config variable (correctly) not found:
params.enable_conda
-
nextflow_config - Config
timeline.enabled
had correct value:true
-
nextflow_config - Config
report.enabled
had correct value:true
-
nextflow_config - Config
trace.enabled
had correct value:true
-
nextflow_config - Config
dag.enabled
had correct value:true
-
nextflow_config - Config
manifest.name
began withnf-core/
-
nextflow_config - Config variable
manifest.homePage
began with https://github.com/nf-core/ -
nextflow_config - Config
dag.file
ended with.html
-
nextflow_config - Config variable
manifest.nextflowVersion
started with >= or !>= -
nextflow_config - Config
manifest.version
ends indev
:3.5.0dev
-
nextflow_config - Config
params.custom_config_version
is set tomaster
-
nextflow_config - Config
params.custom_config_base
is set tohttps://raw.githubusercontent.com/nf-core/configs/master
- nextflow_config - Lines for loading custom profiles found
-
nextflow_config - nextflow.config contains configuration profile
test
- nextflow_config - Config default value correct: params.step= mapping
- nextflow_config - Config default value correct: params.split_fastq= 50000000
- nextflow_config - Config default value correct: params.nucleotides_per_second= 200000
- nextflow_config - Config default value correct: params.clip_r1= 0
- nextflow_config - Config default value correct: params.clip_r2= 0
- nextflow_config - Config default value correct: params.three_prime_clip_r1= 0
- nextflow_config - Config default value correct: params.three_prime_clip_r2= 0
- nextflow_config - Config default value correct: params.trim_nextseq= 0
- nextflow_config - Config default value correct: params.group_by_umi_strategy= Adjacency
- nextflow_config - Config default value correct: params.aligner= bwa-mem
- nextflow_config - Config default value correct: params.ascat_min_base_qual= 20
- nextflow_config - Config default value correct: params.ascat_min_counts= 10
- nextflow_config - Config default value correct: params.ascat_min_map_qual= 35
- nextflow_config - Config default value correct: params.cf_coeff= 0.05
- nextflow_config - Config default value correct: params.cf_contamination= 0
- nextflow_config - Config default value correct: params.cf_minqual= 0
- nextflow_config - Config default value correct: params.cf_mincov= 0
- nextflow_config - Config default value correct: params.cf_ploidy= 2
- nextflow_config - Config default value correct: params.sentieon_haplotyper_emit_mode= variant
- nextflow_config - Config default value correct: params.sentieon_dnascope_emit_mode= variant
- nextflow_config - Config default value correct: params.sentieon_dnascope_pcr_indel_model= CONSERVATIVE
- nextflow_config - Config default value correct: params.dbnsfp_fields= rs_dbSNP,HGVSc_VEP,HGVSp_VEP,1000Gp3_EAS_AF,1000Gp3_AMR_AF,LRT_score,GERP++_RS,gnomAD_exomes_AF
- nextflow_config - Config default value correct: params.vep_custom_args= --everything --filter_common --per_gene --total_length --offline --format vcf
- nextflow_config - Config default value correct: params.vep_version= 111.0-0
- nextflow_config - Config default value correct: params.vep_out_format= vcf
- nextflow_config - Config default value correct: params.genome= GATK.GRCh38
- nextflow_config - Config default value correct: params.igenomes_base= s3://ngi-igenomes/igenomes/
- nextflow_config - Config default value correct: params.vep_cache= s3://annotation-cache/vep_cache/
- nextflow_config - Config default value correct: params.snpeff_cache= s3://annotation-cache/snpeff_cache/
- nextflow_config - Config default value correct: params.custom_config_version= master
- nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
- nextflow_config - Config default value correct: params.test_data_base= https://raw.githubusercontent.com/nf-core/test-datasets/sarek3
- nextflow_config - Config default value correct: params.seq_platform= ILLUMINA
- nextflow_config - Config default value correct: params.max_cpus= 16
- nextflow_config - Config default value correct: params.max_memory= 128.GB
- nextflow_config - Config default value correct: params.max_time= 240.h
- nextflow_config - Config default value correct: params.publish_dir_mode= copy
- nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
- nextflow_config - Config default value correct: params.validate_params= true
- nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
-
files_unchanged -
.gitattributes
matches the template -
files_unchanged -
.prettierrc.yml
matches the template -
files_unchanged -
CODE_OF_CONDUCT.md
matches the template -
files_unchanged -
LICENSE
matches the template -
files_unchanged -
.github/.dockstore.yml
matches the template -
files_unchanged -
.github/CONTRIBUTING.md
matches the template -
files_unchanged -
.github/ISSUE_TEMPLATE/bug_report.yml
matches the template -
files_unchanged -
.github/ISSUE_TEMPLATE/config.yml
matches the template -
files_unchanged -
.github/ISSUE_TEMPLATE/feature_request.yml
matches the template -
files_unchanged -
.github/workflows/branch.yml
matches the template -
files_unchanged -
.github/workflows/linting_comment.yml
matches the template -
files_unchanged -
.github/workflows/linting.yml
matches the template -
files_unchanged -
assets/email_template.html
matches the template -
files_unchanged -
assets/email_template.txt
matches the template -
files_unchanged -
assets/sendmail_template.txt
matches the template -
files_unchanged -
docs/README.md
matches the template -
readme - README Nextflow minimum version badge matched config. Badge:
23.04.0
, Config:23.04.0
- readme - README Zenodo placeholder was replaced with DOI.
- pipeline_name_conventions - Name adheres to nf-core convention
- schema_lint - Schema lint passed
- schema_lint - Schema title + description lint passed
- schema_lint - Input mimetype lint passed: 'text/csv'
- schema_params - Schema matched params returned from nextflow config
-
system_exit - No
System.exit
calls found - actions_schema_validation - Workflow validation passed: ncbench.yml
- actions_schema_validation - Workflow validation passed: branch.yml
- actions_schema_validation - Workflow validation passed: ci.yml
- actions_schema_validation - Workflow validation passed: cloudtest.yml
- actions_schema_validation - Workflow validation passed: clean-up.yml
- actions_schema_validation - Workflow validation passed: fix-linting.yml
- actions_schema_validation - Workflow validation passed: release-announcements.yml
- actions_schema_validation - Workflow validation passed: linting.yml
- actions_schema_validation - Workflow validation passed: linting_comment.yml
- actions_schema_validation - Workflow validation passed: download_pipeline.yml
- merge_markers - No merge markers found in pipeline files
-
modules_json - Only installed modules found in
modules.json
-
multiqc_config -
assets/multiqc_config.yml
found and not ignored. -
multiqc_config -
assets/multiqc_config.yml
containsreport_section_order
-
multiqc_config -
assets/multiqc_config.yml
containsexport_plots
-
multiqc_config -
assets/multiqc_config.yml
containsreport_comment
-
multiqc_config -
assets/multiqc_config.yml
follows the ordering scheme of the minimally required plugins. -
multiqc_config -
assets/multiqc_config.yml
contains a matching 'report_comment'. -
multiqc_config -
assets/multiqc_config.yml
contains 'export_plots: true'. - modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
-
base_config -
conf/base.config
found and not ignored. -
base_config -
UNZIP
found inconf/base.config
and Nextflow scripts. -
base_config -
FASTQC
found inconf/base.config
and Nextflow scripts. -
base_config -
FASTP
found inconf/base.config
and Nextflow scripts. -
base_config -
BWAMEM1_MEM
found inconf/base.config
and Nextflow scripts. -
base_config -
CNVKIT_BATCH
found inconf/base.config
and Nextflow scripts. -
base_config -
GATK4_MARKDUPLICATES
found inconf/base.config
and Nextflow scripts. -
base_config -
GATK4_APPLYBQSR
found inconf/base.config
and Nextflow scripts. -
base_config -
MOSDEPTH
found inconf/base.config
and Nextflow scripts. -
base_config -
STRELKA
found inconf/base.config
and Nextflow scripts. -
base_config -
SAMTOOLS_CONVERT
found inconf/base.config
and Nextflow scripts. -
base_config -
GATK4_MERGEVCFS
found inconf/base.config
and Nextflow scripts. -
base_config -
MULTIQC
found inconf/base.config
and Nextflow scripts. -
nfcore_yml - Repository type in
.nf-core.yml
is valid:pipeline
-
nfcore_yml - nf-core version in
.nf-core.yml
is set to the latest version:2.14.1
Run details
- nf-core/tools version 2.14.1
- Run at
2024-05-11 13:39:52
proposals for docs and changelog updates just submitted. feel free to modify :)
Looks very good 👏👏 What about CI-tests covering this new functionality?
I've added 2 emits for the raw and exported CNV calls. However, at the moment there's no merging of SNP/INDEL variant calls and CNV calls into the variant calling subworkflow(s). This is a feature requested by users, to my understanding. This emit will ensure being able to add this in the future, but unless some additional work is done on the bam_variant_calling_* subworkflows, the emitted channels are not used at the moment. This is beyond the scope of this PR though :)
Thanks for the updates 🙌
Concerning CI-tests for this new functionality, I see we already have this pytest of the germline subworkflow of cnvkit, and it runs CNVKIT_EXPORT
and generates results/variant_calling/cnvkit/sample1/sample1.cnvcall.vcf
.
Currently that pytest doesn't check for the existence of sample1.cnvcall.vcf
- perhaps it should?
EDIT: Perhaps also check for the existence of results/variant_calling/cnvkit/sample1/test.paired_end.recalibrated.sorted.germline.call.cns
which seems to be generated by CNVKIT_CALL
in that pytest?