TMB
#495
Coming up with a POC for the tmb support:
Idea is to keep all relevant config files for the tmb tool in our own assests folder (or any other that we can control for now). Otherwise we are dependent that all of the tools that we would like to support will get added & released upstream. (Over time of course we could contribute them).
For all vcf files the variantcaller & annotation tool needs to be tracked in the meta map. We are already doing this for the variantcallers anyways. So this should be a small and easy change.
Last but not least the respective file can be retrieved with a groovy function for the proper combination.
Now of course we need to start generating all the db and var config files.
I am not sure yet how we could possibly handle users overwritting the config file. From where I stand at the moment we would need a parameter with which you can pass multiple files with a key...
Hi @FriederikeHanssen,
I had a look at your code ! that's great.
For information, there is a typo in the tmb module
--dbConfig ${dbConfig} \
should be
--dbConfig ${dbconfig} \
with small 'c'
nf-core lint overall result: Passed :white_check_mark: :warning:
Posted for pipeline commit 44d56a1
+| ✅ 151 tests passed |+
#| ❔ 8 tests were ignored |#
!| ❗ 1 tests had warnings |!
:heavy_exclamation_mark: Test warnings:
- pipeline_todos - TODO string in
methods_description_template.yml: #Update the HTML below to your prefered methods description, e.g. add publication citation for this pipeline
:grey_question: Tests ignored:
- files_exist - File is ignored:
conf/modules.config - files_exist - File is ignored:
conf/test.config - files_exist - File is ignored:
conf/test_full.config - files_unchanged - File ignored due to lint config:
assets/nf-core-sarek_logo_light.png - files_unchanged - File ignored due to lint config:
docs/images/nf-core-sarek_logo_light.png - files_unchanged - File ignored due to lint config:
docs/images/nf-core-sarek_logo_dark.png - files_unchanged - File ignored due to lint config:
lib/NfcoreTemplate.groovy - template_strings - template_strings
:white_check_mark: Tests passed:
- files_exist - File found:
.gitattributes - files_exist - File found:
.gitignore - files_exist - File found:
.nf-core.yml - files_exist - File found:
.editorconfig - files_exist - File found:
.prettierignore - files_exist - File found:
.prettierrc.yml - files_exist - File found:
CHANGELOG.md - files_exist - File found:
CITATIONS.md - files_exist - File found:
CODE_OF_CONDUCT.md - files_exist - File found:
CODE_OF_CONDUCT.md - files_exist - File found:
LICENSEorLICENSE.mdorLICENCEorLICENCE.md - files_exist - File found:
nextflow_schema.json - files_exist - File found:
nextflow.config - files_exist - File found:
README.md - files_exist - File found:
.github/.dockstore.yml - files_exist - File found:
.github/CONTRIBUTING.md - files_exist - File found:
.github/ISSUE_TEMPLATE/bug_report.yml - files_exist - File found:
.github/ISSUE_TEMPLATE/config.yml - files_exist - File found:
.github/ISSUE_TEMPLATE/feature_request.yml - files_exist - File found:
.github/PULL_REQUEST_TEMPLATE.md - files_exist - File found:
.github/workflows/branch.yml - files_exist - File found:
.github/workflows/ci.yml - files_exist - File found:
.github/workflows/linting_comment.yml - files_exist - File found:
.github/workflows/linting.yml - files_exist - File found:
assets/email_template.html - files_exist - File found:
assets/email_template.txt - files_exist - File found:
assets/sendmail_template.txt - files_exist - File found:
assets/nf-core-sarek_logo_light.png - files_exist - File found:
docs/images/nf-core-sarek_logo_light.png - files_exist - File found:
docs/images/nf-core-sarek_logo_dark.png - files_exist - File found:
docs/output.md - files_exist - File found:
docs/README.md - files_exist - File found:
docs/README.md - files_exist - File found:
docs/usage.md - files_exist - File found:
lib/nfcore_external_java_deps.jar - files_exist - File found:
lib/NfcoreSchema.groovy - files_exist - File found:
lib/NfcoreTemplate.groovy - files_exist - File found:
lib/Utils.groovy - files_exist - File found:
lib/WorkflowMain.groovy - files_exist - File found:
main.nf - files_exist - File found:
assets/multiqc_config.yml - files_exist - File found:
conf/base.config - files_exist - File found:
conf/igenomes.config - files_exist - File found:
.github/workflows/awstest.yml - files_exist - File found:
.github/workflows/awsfulltest.yml - files_exist - File found:
lib/WorkflowSarek.groovy - files_exist - File found:
modules.json - files_exist - File found:
pyproject.toml - files_exist - File not found check:
Singularity - files_exist - File not found check:
parameters.settings.json - files_exist - File not found check:
.nf-core.yaml - files_exist - File not found check:
bin/markdown_to_html.r - files_exist - File not found check:
conf/aws.config - files_exist - File not found check:
.github/workflows/push_dockerhub.yml - files_exist - File not found check:
.github/ISSUE_TEMPLATE/bug_report.md - files_exist - File not found check:
.github/ISSUE_TEMPLATE/feature_request.md - files_exist - File not found check:
docs/images/nf-core-sarek_logo.png - files_exist - File not found check:
.markdownlint.yml - files_exist - File not found check:
.yamllint.yml - files_exist - File not found check:
lib/Checks.groovy - files_exist - File not found check:
lib/Completion.groovy - files_exist - File not found check:
lib/Workflow.groovy - files_exist - File not found check:
.travis.yml - nextflow_config - Config variable found:
manifest.name - nextflow_config - Config variable found:
manifest.nextflowVersion - nextflow_config - Config variable found:
manifest.description - nextflow_config - Config variable found:
manifest.version - nextflow_config - Config variable found:
manifest.homePage - nextflow_config - Config variable found:
timeline.enabled - nextflow_config - Config variable found:
trace.enabled - nextflow_config - Config variable found:
report.enabled - nextflow_config - Config variable found:
dag.enabled - nextflow_config - Config variable found:
process.cpus - nextflow_config - Config variable found:
process.memory - nextflow_config - Config variable found:
process.time - nextflow_config - Config variable found:
params.outdir - nextflow_config - Config variable found:
params.input - nextflow_config - Config variable found:
params.show_hidden_params - nextflow_config - Config variable found:
params.schema_ignore_params - nextflow_config - Config variable found:
manifest.mainScript - nextflow_config - Config variable found:
timeline.file - nextflow_config - Config variable found:
trace.file - nextflow_config - Config variable found:
report.file - nextflow_config - Config variable found:
dag.file - nextflow_config - Config variable (correctly) not found:
params.version - nextflow_config - Config variable (correctly) not found:
params.nf_required_version - nextflow_config - Config variable (correctly) not found:
params.container - nextflow_config - Config variable (correctly) not found:
params.singleEnd - nextflow_config - Config variable (correctly) not found:
params.igenomesIgnore - nextflow_config - Config variable (correctly) not found:
params.name - nextflow_config - Config
timeline.enabledhad correct value:true - nextflow_config - Config
report.enabledhad correct value:true - nextflow_config - Config
trace.enabledhad correct value:true - nextflow_config - Config
dag.enabledhad correct value:true - nextflow_config - Config
manifest.namebegan withnf-core/ - nextflow_config - Config variable
manifest.homePagebegan with https://github.com/nf-core/ - nextflow_config - Config
dag.fileended with.html - nextflow_config - Config variable
manifest.nextflowVersionstarted with >= or !>= - nextflow_config - Config
manifest.versionends indev:'3.1dev' - nextflow_config - Config
params.custom_config_versionis set tomaster - nextflow_config - Config
params.custom_config_baseis set tohttps://raw.githubusercontent.com/nf-core/configs/master - nextflow_config - Lines for loading custom profiles found
- files_unchanged -
.gitattributesmatches the template - files_unchanged -
.prettierrc.ymlmatches the template - files_unchanged -
CODE_OF_CONDUCT.mdmatches the template - files_unchanged -
LICENSEmatches the template - files_unchanged -
.github/.dockstore.ymlmatches the template - files_unchanged -
.github/CONTRIBUTING.mdmatches the template - files_unchanged -
.github/ISSUE_TEMPLATE/bug_report.ymlmatches the template - files_unchanged -
.github/ISSUE_TEMPLATE/config.ymlmatches the template - files_unchanged -
.github/ISSUE_TEMPLATE/feature_request.ymlmatches the template - files_unchanged -
.github/PULL_REQUEST_TEMPLATE.mdmatches the template - files_unchanged -
.github/workflows/branch.ymlmatches the template - files_unchanged -
.github/workflows/linting_comment.ymlmatches the template - files_unchanged -
.github/workflows/linting.ymlmatches the template - files_unchanged -
assets/email_template.htmlmatches the template - files_unchanged -
assets/email_template.txtmatches the template - files_unchanged -
assets/sendmail_template.txtmatches the template - files_unchanged -
docs/README.mdmatches the template - files_unchanged -
lib/nfcore_external_java_deps.jarmatches the template - files_unchanged -
lib/NfcoreSchema.groovymatches the template - files_unchanged -
.gitignorematches the template - files_unchanged -
.prettierignorematches the template - files_unchanged -
pyproject.tomlmatches the template - actions_ci - '.github/workflows/ci.yml' is triggered on expected events
- actions_ci - '.github/workflows/ci.yml' checks minimum NF version
- actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
- actions_awsfulltest -
.github/workflows/awsfulltest.ymlis triggered correctly - actions_awsfulltest -
.github/workflows/awsfulltest.ymldoes not use-profile test - readme - README Nextflow minimum version badge matched config. Badge:
21.10.3, Config:21.10.3 - readme - README Nextflow minimum version in Quick Start section matched config. README:
21.10.3, Config:21.10.3 - pipeline_name_conventions - Name adheres to nf-core convention
- schema_lint - Schema lint passed
- schema_lint - Schema title + description lint passed
- schema_lint - Input mimetype lint passed: 'text/csv'
- schema_params - Schema matched params returned from nextflow config
- actions_schema_validation - Workflow validation passed: awsfulltest.yml
- actions_schema_validation - Workflow validation passed: ci.yml
- actions_schema_validation - Workflow validation passed: linting.yml
- actions_schema_validation - Workflow validation passed: pytest-workflow.yml
- actions_schema_validation - Workflow validation passed: linting_comment.yml
- actions_schema_validation - Workflow validation passed: fix-linting.yml
- actions_schema_validation - Workflow validation passed: awsfulltest_germline.yml
- actions_schema_validation - Workflow validation passed: branch.yml
- actions_schema_validation - Workflow validation passed: awstest.yml
- merge_markers - No merge markers found in pipeline files
- modules_json - Only installed modules found in
modules.json - multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
- multiqc_config - 'assets/multiqc_config.yml' contains a matching 'report_comment'.
- multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
- modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
Run details
- nf-core/tools version 2.6
- Run at
2022-11-10 23:19:32
- [ ] Update module in nf-core/modules
- [ ] vep.yml
- [ ] strelka.yml
- [ ] freebayes.yml
- [ ] Clear up effGenomeSize computation
- [ ] Update how we handle intervals when running annotation
- [ ] input starting from tmb directly also needs tbi, but for annotation this would cause issues with channels....
- [ ] need to propagate all tbis up from variantcalling
- [ ] allow input configs?
Ok this is starting to take shape. A couple of things that I could use help with:
- generating the yml files for VEP, Strelka, & Freebayes.
- Which parameters do we need?
1 & 2 maybe @apeltzer @tomgutman have any advice?
- @maxulysse this is starting to shape into a new subworkflow after annotation, how do we want to call this step? post-processing is not descriptive. We would probably need to track back metadata that we dropped. Here for example we only want to run this tool on somatic data
For 3.) --> maybe somehing like "downstream-metrics"? :-)
For 3.) --> maybe somehing like "downstream-metrics"? :-)
I'd even go just for downstream
Hello,
unfortunately I don't have any expertise in VEP, strelka and freebayes. So I can't really help on that. =/ Sorry
I'll prio the generation of the YAML files soon and will either find some time myself or have someone look into it.