sarek
sarek copied to clipboard
TMB
#495
Coming up with a POC for the tmb support:
Idea is to keep all relevant config files for the tmb tool in our own assests folder (or any other that we can control for now). Otherwise we are dependent that all of the tools that we would like to support will get added & released upstream. (Over time of course we could contribute them).
For all vcf files the variantcaller & annotation tool needs to be tracked in the meta map. We are already doing this for the variantcallers anyways. So this should be a small and easy change.
Last but not least the respective file can be retrieved with a groovy function for the proper combination.
Now of course we need to start generating all the db and var config files.
I am not sure yet how we could possibly handle users overwritting the config file. From where I stand at the moment we would need a parameter with which you can pass multiple files with a key...
Hi @FriederikeHanssen,
I had a look at your code ! that's great.
For information, there is a typo in the tmb
module
--dbConfig ${dbConfig} \
should be
--dbConfig ${dbconfig} \
with small 'c'
nf-core lint
overall result: Passed :white_check_mark: :warning:
Posted for pipeline commit 44d56a1
+| ✅ 151 tests passed |+
#| ❔ 8 tests were ignored |#
!| ❗ 1 tests had warnings |!
:heavy_exclamation_mark: Test warnings:
-
pipeline_todos - TODO string in
methods_description_template.yml
: #Update the HTML below to your prefered methods description, e.g. add publication citation for this pipeline
:grey_question: Tests ignored:
-
files_exist - File is ignored:
conf/modules.config
-
files_exist - File is ignored:
conf/test.config
-
files_exist - File is ignored:
conf/test_full.config
-
files_unchanged - File ignored due to lint config:
assets/nf-core-sarek_logo_light.png
-
files_unchanged - File ignored due to lint config:
docs/images/nf-core-sarek_logo_light.png
-
files_unchanged - File ignored due to lint config:
docs/images/nf-core-sarek_logo_dark.png
-
files_unchanged - File ignored due to lint config:
lib/NfcoreTemplate.groovy
- template_strings - template_strings
:white_check_mark: Tests passed:
-
files_exist - File found:
.gitattributes
-
files_exist - File found:
.gitignore
-
files_exist - File found:
.nf-core.yml
-
files_exist - File found:
.editorconfig
-
files_exist - File found:
.prettierignore
-
files_exist - File found:
.prettierrc.yml
-
files_exist - File found:
CHANGELOG.md
-
files_exist - File found:
CITATIONS.md
-
files_exist - File found:
CODE_OF_CONDUCT.md
-
files_exist - File found:
CODE_OF_CONDUCT.md
-
files_exist - File found:
LICENSE
orLICENSE.md
orLICENCE
orLICENCE.md
-
files_exist - File found:
nextflow_schema.json
-
files_exist - File found:
nextflow.config
-
files_exist - File found:
README.md
-
files_exist - File found:
.github/.dockstore.yml
-
files_exist - File found:
.github/CONTRIBUTING.md
-
files_exist - File found:
.github/ISSUE_TEMPLATE/bug_report.yml
-
files_exist - File found:
.github/ISSUE_TEMPLATE/config.yml
-
files_exist - File found:
.github/ISSUE_TEMPLATE/feature_request.yml
-
files_exist - File found:
.github/PULL_REQUEST_TEMPLATE.md
-
files_exist - File found:
.github/workflows/branch.yml
-
files_exist - File found:
.github/workflows/ci.yml
-
files_exist - File found:
.github/workflows/linting_comment.yml
-
files_exist - File found:
.github/workflows/linting.yml
-
files_exist - File found:
assets/email_template.html
-
files_exist - File found:
assets/email_template.txt
-
files_exist - File found:
assets/sendmail_template.txt
-
files_exist - File found:
assets/nf-core-sarek_logo_light.png
-
files_exist - File found:
docs/images/nf-core-sarek_logo_light.png
-
files_exist - File found:
docs/images/nf-core-sarek_logo_dark.png
-
files_exist - File found:
docs/output.md
-
files_exist - File found:
docs/README.md
-
files_exist - File found:
docs/README.md
-
files_exist - File found:
docs/usage.md
-
files_exist - File found:
lib/nfcore_external_java_deps.jar
-
files_exist - File found:
lib/NfcoreSchema.groovy
-
files_exist - File found:
lib/NfcoreTemplate.groovy
-
files_exist - File found:
lib/Utils.groovy
-
files_exist - File found:
lib/WorkflowMain.groovy
-
files_exist - File found:
main.nf
-
files_exist - File found:
assets/multiqc_config.yml
-
files_exist - File found:
conf/base.config
-
files_exist - File found:
conf/igenomes.config
-
files_exist - File found:
.github/workflows/awstest.yml
-
files_exist - File found:
.github/workflows/awsfulltest.yml
-
files_exist - File found:
lib/WorkflowSarek.groovy
-
files_exist - File found:
modules.json
-
files_exist - File found:
pyproject.toml
-
files_exist - File not found check:
Singularity
-
files_exist - File not found check:
parameters.settings.json
-
files_exist - File not found check:
.nf-core.yaml
-
files_exist - File not found check:
bin/markdown_to_html.r
-
files_exist - File not found check:
conf/aws.config
-
files_exist - File not found check:
.github/workflows/push_dockerhub.yml
-
files_exist - File not found check:
.github/ISSUE_TEMPLATE/bug_report.md
-
files_exist - File not found check:
.github/ISSUE_TEMPLATE/feature_request.md
-
files_exist - File not found check:
docs/images/nf-core-sarek_logo.png
-
files_exist - File not found check:
.markdownlint.yml
-
files_exist - File not found check:
.yamllint.yml
-
files_exist - File not found check:
lib/Checks.groovy
-
files_exist - File not found check:
lib/Completion.groovy
-
files_exist - File not found check:
lib/Workflow.groovy
-
files_exist - File not found check:
.travis.yml
-
nextflow_config - Config variable found:
manifest.name
-
nextflow_config - Config variable found:
manifest.nextflowVersion
-
nextflow_config - Config variable found:
manifest.description
-
nextflow_config - Config variable found:
manifest.version
-
nextflow_config - Config variable found:
manifest.homePage
-
nextflow_config - Config variable found:
timeline.enabled
-
nextflow_config - Config variable found:
trace.enabled
-
nextflow_config - Config variable found:
report.enabled
-
nextflow_config - Config variable found:
dag.enabled
-
nextflow_config - Config variable found:
process.cpus
-
nextflow_config - Config variable found:
process.memory
-
nextflow_config - Config variable found:
process.time
-
nextflow_config - Config variable found:
params.outdir
-
nextflow_config - Config variable found:
params.input
-
nextflow_config - Config variable found:
params.show_hidden_params
-
nextflow_config - Config variable found:
params.schema_ignore_params
-
nextflow_config - Config variable found:
manifest.mainScript
-
nextflow_config - Config variable found:
timeline.file
-
nextflow_config - Config variable found:
trace.file
-
nextflow_config - Config variable found:
report.file
-
nextflow_config - Config variable found:
dag.file
-
nextflow_config - Config variable (correctly) not found:
params.version
-
nextflow_config - Config variable (correctly) not found:
params.nf_required_version
-
nextflow_config - Config variable (correctly) not found:
params.container
-
nextflow_config - Config variable (correctly) not found:
params.singleEnd
-
nextflow_config - Config variable (correctly) not found:
params.igenomesIgnore
-
nextflow_config - Config variable (correctly) not found:
params.name
-
nextflow_config - Config
timeline.enabled
had correct value:true
-
nextflow_config - Config
report.enabled
had correct value:true
-
nextflow_config - Config
trace.enabled
had correct value:true
-
nextflow_config - Config
dag.enabled
had correct value:true
-
nextflow_config - Config
manifest.name
began withnf-core/
-
nextflow_config - Config variable
manifest.homePage
began with https://github.com/nf-core/ -
nextflow_config - Config
dag.file
ended with.html
-
nextflow_config - Config variable
manifest.nextflowVersion
started with >= or !>= -
nextflow_config - Config
manifest.version
ends indev
:'3.1dev'
-
nextflow_config - Config
params.custom_config_version
is set tomaster
-
nextflow_config - Config
params.custom_config_base
is set tohttps://raw.githubusercontent.com/nf-core/configs/master
- nextflow_config - Lines for loading custom profiles found
-
files_unchanged -
.gitattributes
matches the template -
files_unchanged -
.prettierrc.yml
matches the template -
files_unchanged -
CODE_OF_CONDUCT.md
matches the template -
files_unchanged -
LICENSE
matches the template -
files_unchanged -
.github/.dockstore.yml
matches the template -
files_unchanged -
.github/CONTRIBUTING.md
matches the template -
files_unchanged -
.github/ISSUE_TEMPLATE/bug_report.yml
matches the template -
files_unchanged -
.github/ISSUE_TEMPLATE/config.yml
matches the template -
files_unchanged -
.github/ISSUE_TEMPLATE/feature_request.yml
matches the template -
files_unchanged -
.github/PULL_REQUEST_TEMPLATE.md
matches the template -
files_unchanged -
.github/workflows/branch.yml
matches the template -
files_unchanged -
.github/workflows/linting_comment.yml
matches the template -
files_unchanged -
.github/workflows/linting.yml
matches the template -
files_unchanged -
assets/email_template.html
matches the template -
files_unchanged -
assets/email_template.txt
matches the template -
files_unchanged -
assets/sendmail_template.txt
matches the template -
files_unchanged -
docs/README.md
matches the template -
files_unchanged -
lib/nfcore_external_java_deps.jar
matches the template -
files_unchanged -
lib/NfcoreSchema.groovy
matches the template -
files_unchanged -
.gitignore
matches the template -
files_unchanged -
.prettierignore
matches the template -
files_unchanged -
pyproject.toml
matches the template - actions_ci - '.github/workflows/ci.yml' is triggered on expected events
- actions_ci - '.github/workflows/ci.yml' checks minimum NF version
- actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
-
actions_awsfulltest -
.github/workflows/awsfulltest.yml
is triggered correctly -
actions_awsfulltest -
.github/workflows/awsfulltest.yml
does not use-profile test
-
readme - README Nextflow minimum version badge matched config. Badge:
21.10.3
, Config:21.10.3
-
readme - README Nextflow minimum version in Quick Start section matched config. README:
21.10.3
, Config:21.10.3
- pipeline_name_conventions - Name adheres to nf-core convention
- schema_lint - Schema lint passed
- schema_lint - Schema title + description lint passed
- schema_lint - Input mimetype lint passed: 'text/csv'
- schema_params - Schema matched params returned from nextflow config
- actions_schema_validation - Workflow validation passed: awsfulltest.yml
- actions_schema_validation - Workflow validation passed: ci.yml
- actions_schema_validation - Workflow validation passed: linting.yml
- actions_schema_validation - Workflow validation passed: pytest-workflow.yml
- actions_schema_validation - Workflow validation passed: linting_comment.yml
- actions_schema_validation - Workflow validation passed: fix-linting.yml
- actions_schema_validation - Workflow validation passed: awsfulltest_germline.yml
- actions_schema_validation - Workflow validation passed: branch.yml
- actions_schema_validation - Workflow validation passed: awstest.yml
- merge_markers - No merge markers found in pipeline files
-
modules_json - Only installed modules found in
modules.json
- multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
- multiqc_config - 'assets/multiqc_config.yml' contains a matching 'report_comment'.
- multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
- modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
Run details
- nf-core/tools version 2.6
- Run at
2022-11-10 23:19:32
- [ ] Update module in nf-core/modules
- [ ] vep.yml
- [ ] strelka.yml
- [ ] freebayes.yml
- [ ] Clear up effGenomeSize computation
- [ ] Update how we handle intervals when running annotation
- [ ] input starting from tmb directly also needs tbi, but for annotation this would cause issues with channels....
- [ ] need to propagate all tbis up from variantcalling
- [ ] allow input configs?
Ok this is starting to take shape. A couple of things that I could use help with:
- generating the yml files for VEP, Strelka, & Freebayes.
- Which parameters do we need?
1 & 2 maybe @apeltzer @tomgutman have any advice?
- @maxulysse this is starting to shape into a new subworkflow after annotation, how do we want to call this step? post-processing is not descriptive. We would probably need to track back metadata that we dropped. Here for example we only want to run this tool on somatic data
For 3.) --> maybe somehing like "downstream-metrics"? :-)
For 3.) --> maybe somehing like "downstream-metrics"? :-)
I'd even go just for downstream
Hello,
unfortunately I don't have any expertise in VEP, strelka and freebayes. So I can't really help on that. =/ Sorry
I'll prio the generation of the YAML files soon and will either find some time myself or have someone look into it.