Improved ext.args consolidation for STAR and TRIMGALORE
Occasionally, people were surprised that parameters provided to extra_star_align_args could still conflict with the default configuration (e.g. #1046).
As a convenience parameter, I felt that it should never take precedence over the actual module configuration, because it seemed more error-prone to silently overwrite the module config with something that (as the parameter name suggests) is meant to be extra. Hence, I preferred that a custom module config should be used.
However, I said that I would be open to reconsider and since now also @tdanhorn and @CharlotteAnne suggested that change, I gave in. This PR now improves the extra_args parameter consolidation so it can override presets from the config.
PR checklist
- [x] This comment contains a description of changes (with reason).
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [x] Make sure your code lints (
nf-core lint). - [x] Ensure the test suite passes (
nextflow run . -profile test,docker --outdir <OUTDIR>). - [ ] Check for unexpected warnings in debug mode (
nextflow run . -profile debug,test,docker --outdir <OUTDIR>). - [x]
CHANGELOG.mdis updated.
nf-core lint overall result: Passed :white_check_mark: :warning:
Posted for pipeline commit 6ab2e44
+| ✅ 167 tests passed |+
#| ❔ 9 tests were ignored |#
!| ❗ 7 tests had warnings |!
:heavy_exclamation_mark: Test warnings:
- files_exist - File not found:
assets/multiqc_config.yml - files_exist - File not found:
.github/workflows/awstest.yml - files_exist - File not found:
.github/workflows/awsfulltest.yml - pipeline_todos - TODO string in
methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline - pipeline_todos - TODO string in
main.nf: Optionally add in-text citation tools to this list. - pipeline_todos - TODO string in
main.nf: Optionally add bibliographic entries to this list. - pipeline_todos - TODO string in
main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
:grey_question: Tests ignored:
- files_exist - File is ignored:
conf/modules.config - nextflow_config - Config default ignored: params.ribo_database_manifest
- files_unchanged - File ignored due to lint config:
.github/PULL_REQUEST_TEMPLATE.md - files_unchanged - File ignored due to lint config:
assets/email_template.html - files_unchanged - File ignored due to lint config:
assets/email_template.txt - files_unchanged - File ignored due to lint config:
.gitignoreor.prettierignoreorpyproject.toml - actions_ci - actions_ci
- actions_awstest - 'awstest.yml' workflow not found:
/home/runner/work/rnaseq/rnaseq/.github/workflows/awstest.yml - multiqc_config - 'assets/multiqc_config.yml' not found
:white_check_mark: Tests passed:
- files_exist - File found:
.gitattributes - files_exist - File found:
.gitignore - files_exist - File found:
.nf-core.yml - files_exist - File found:
.editorconfig - files_exist - File found:
.prettierignore - files_exist - File found:
.prettierrc.yml - files_exist - File found:
CHANGELOG.md - files_exist - File found:
CITATIONS.md - files_exist - File found:
CODE_OF_CONDUCT.md - files_exist - File found:
LICENSEorLICENSE.mdorLICENCEorLICENCE.md - files_exist - File found:
nextflow_schema.json - files_exist - File found:
nextflow.config - files_exist - File found:
README.md - files_exist - File found:
.github/.dockstore.yml - files_exist - File found:
.github/CONTRIBUTING.md - files_exist - File found:
.github/ISSUE_TEMPLATE/bug_report.yml - files_exist - File found:
.github/ISSUE_TEMPLATE/config.yml - files_exist - File found:
.github/ISSUE_TEMPLATE/feature_request.yml - files_exist - File found:
.github/PULL_REQUEST_TEMPLATE.md - files_exist - File found:
.github/workflows/branch.yml - files_exist - File found:
.github/workflows/ci.yml - files_exist - File found:
.github/workflows/linting_comment.yml - files_exist - File found:
.github/workflows/linting.yml - files_exist - File found:
assets/email_template.html - files_exist - File found:
assets/email_template.txt - files_exist - File found:
assets/sendmail_template.txt - files_exist - File found:
assets/nf-core-rnaseq_logo_light.png - files_exist - File found:
conf/test.config - files_exist - File found:
conf/test_full.config - files_exist - File found:
docs/images/nf-core-rnaseq_logo_light.png - files_exist - File found:
docs/images/nf-core-rnaseq_logo_dark.png - files_exist - File found:
docs/output.md - files_exist - File found:
docs/README.md - files_exist - File found:
docs/README.md - files_exist - File found:
docs/usage.md - files_exist - File found:
main.nf - files_exist - File found:
conf/base.config - files_exist - File found:
conf/igenomes.config - files_exist - File found:
modules.json - files_exist - File found:
pyproject.toml - files_exist - File not found check:
Singularity - files_exist - File not found check:
parameters.settings.json - files_exist - File not found check:
pipeline_template.yml - files_exist - File not found check:
.nf-core.yaml - files_exist - File not found check:
bin/markdown_to_html.r - files_exist - File not found check:
conf/aws.config - files_exist - File not found check:
.github/workflows/push_dockerhub.yml - files_exist - File not found check:
.github/ISSUE_TEMPLATE/bug_report.md - files_exist - File not found check:
.github/ISSUE_TEMPLATE/feature_request.md - files_exist - File not found check:
docs/images/nf-core-rnaseq_logo.png - files_exist - File not found check:
.markdownlint.yml - files_exist - File not found check:
.yamllint.yml - files_exist - File not found check:
lib/Checks.groovy - files_exist - File not found check:
lib/Completion.groovy - files_exist - File not found check:
lib/Workflow.groovy - files_exist - File not found check:
lib/Utils.groovy - files_exist - File not found check:
lib/WorkflowMain.groovy - files_exist - File not found check:
lib/NfcoreTemplate.groovy - files_exist - File not found check:
lib/WorkflowRnaseq.groovy - files_exist - File not found check:
lib/nfcore_external_java_deps.jar - files_exist - File not found check:
.travis.yml - nextflow_config - Config variable found:
manifest.name - nextflow_config - Config variable found:
manifest.nextflowVersion - nextflow_config - Config variable found:
manifest.description - nextflow_config - Config variable found:
manifest.version - nextflow_config - Config variable found:
manifest.homePage - nextflow_config - Config variable found:
timeline.enabled - nextflow_config - Config variable found:
trace.enabled - nextflow_config - Config variable found:
report.enabled - nextflow_config - Config variable found:
dag.enabled - nextflow_config - Config variable found:
process.cpus - nextflow_config - Config variable found:
process.memory - nextflow_config - Config variable found:
process.time - nextflow_config - Config variable found:
params.outdir - nextflow_config - Config variable found:
params.input - nextflow_config - Config variable found:
params.validationShowHiddenParams - nextflow_config - Config variable found:
params.validationSchemaIgnoreParams - nextflow_config - Config variable found:
manifest.mainScript - nextflow_config - Config variable found:
timeline.file - nextflow_config - Config variable found:
trace.file - nextflow_config - Config variable found:
report.file - nextflow_config - Config variable found:
dag.file - nextflow_config - Config variable (correctly) not found:
params.nf_required_version - nextflow_config - Config variable (correctly) not found:
params.container - nextflow_config - Config variable (correctly) not found:
params.singleEnd - nextflow_config - Config variable (correctly) not found:
params.igenomesIgnore - nextflow_config - Config variable (correctly) not found:
params.name - nextflow_config - Config variable (correctly) not found:
params.enable_conda - nextflow_config - Config
timeline.enabledhad correct value:true - nextflow_config - Config
report.enabledhad correct value:true - nextflow_config - Config
trace.enabledhad correct value:true - nextflow_config - Config
dag.enabledhad correct value:true - nextflow_config - Config
manifest.namebegan withnf-core/ - nextflow_config - Config variable
manifest.homePagebegan with https://github.com/nf-core/ - nextflow_config - Config
dag.fileended with.html - nextflow_config - Config variable
manifest.nextflowVersionstarted with >= or !>= - nextflow_config - Config
manifest.versionends indev:3.15.0dev - nextflow_config - Config
params.custom_config_versionis set tomaster - nextflow_config - Config
params.custom_config_baseis set tohttps://raw.githubusercontent.com/nf-core/configs/master - nextflow_config - Lines for loading custom profiles found
- nextflow_config - nextflow.config contains configuration profile
test - nextflow_config - Config default value correct: params.hisat2_build_memory= 200.GB
- nextflow_config - Config default value correct: params.gtf_extra_attributes= gene_name
- nextflow_config - Config default value correct: params.gtf_group_features= gene_id
- nextflow_config - Config default value correct: params.featurecounts_group_type= gene_biotype
- nextflow_config - Config default value correct: params.featurecounts_feature_type= exon
- nextflow_config - Config default value correct: params.igenomes_base= s3://ngi-igenomes/igenomes
- nextflow_config - Config default value correct: params.trimmer= trimgalore
- nextflow_config - Config default value correct: params.min_trimmed_reads= 10000
- nextflow_config - Config default value correct: params.umitools_extract_method= string
- nextflow_config - Config default value correct: params.umitools_grouping_method= directional
- nextflow_config - Config default value correct: params.aligner= star_salmon
- nextflow_config - Config default value correct: params.pseudo_aligner_kmer_size= 31
- nextflow_config - Config default value correct: params.min_mapped_reads= 5.0
- nextflow_config - Config default value correct: params.kallisto_quant_fraglen= 200
- nextflow_config - Config default value correct: params.kallisto_quant_fraglen_sd= 200
- nextflow_config - Config default value correct: params.deseq2_vst= true
- nextflow_config - Config default value correct: params.rseqc_modules= bam_stat,inner_distance,infer_experiment,junction_annotation,junction_saturation,read_distribution,read_duplication
- nextflow_config - Config default value correct: params.skip_bbsplit= true
- nextflow_config - Config default value correct: params.skip_preseq= true
- nextflow_config - Config default value correct: params.custom_config_version= master
- nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
- nextflow_config - Config default value correct: params.max_cpus= 16
- nextflow_config - Config default value correct: params.max_memory= 128.GB
- nextflow_config - Config default value correct: params.max_time= 240.h
- nextflow_config - Config default value correct: params.publish_dir_mode= copy
- nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
- nextflow_config - Config default value correct: params.validate_params= true
- files_unchanged -
.gitattributesmatches the template - files_unchanged -
.prettierrc.ymlmatches the template - files_unchanged -
CODE_OF_CONDUCT.mdmatches the template - files_unchanged -
LICENSEmatches the template - files_unchanged -
.github/.dockstore.ymlmatches the template - files_unchanged -
.github/CONTRIBUTING.mdmatches the template - files_unchanged -
.github/ISSUE_TEMPLATE/bug_report.ymlmatches the template - files_unchanged -
.github/ISSUE_TEMPLATE/config.ymlmatches the template - files_unchanged -
.github/ISSUE_TEMPLATE/feature_request.ymlmatches the template - files_unchanged -
.github/workflows/branch.ymlmatches the template - files_unchanged -
.github/workflows/linting_comment.ymlmatches the template - files_unchanged -
.github/workflows/linting.ymlmatches the template - files_unchanged -
assets/sendmail_template.txtmatches the template - files_unchanged -
assets/nf-core-rnaseq_logo_light.pngmatches the template - files_unchanged -
docs/images/nf-core-rnaseq_logo_light.pngmatches the template - files_unchanged -
docs/images/nf-core-rnaseq_logo_dark.pngmatches the template - files_unchanged -
docs/README.mdmatches the template - readme - README Nextflow minimum version badge matched config. Badge:
23.04.0, Config:23.04.0 - readme - README Zenodo placeholder was replaced with DOI.
- pipeline_name_conventions - Name adheres to nf-core convention
- template_strings - Did not find any Jinja template strings (540 files)
- schema_lint - Schema lint passed
- schema_lint - Schema title + description lint passed
- schema_lint - Input mimetype lint passed: 'text/csv'
- schema_params - Schema matched params returned from nextflow config
- system_exit - No
System.exitcalls found - actions_schema_validation - Workflow validation passed: branch.yml
- actions_schema_validation - Workflow validation passed: ci.yml
- actions_schema_validation - Workflow validation passed: clean-up.yml
- actions_schema_validation - Workflow validation passed: fix-linting.yml
- actions_schema_validation - Workflow validation passed: cloud_tests_small.yml
- actions_schema_validation - Workflow validation passed: release-announcements.yml
- actions_schema_validation - Workflow validation passed: linting.yml
- actions_schema_validation - Workflow validation passed: cloud_tests_full.yml
- actions_schema_validation - Workflow validation passed: linting_comment.yml
- actions_schema_validation - Workflow validation passed: download_pipeline.yml
- merge_markers - No merge markers found in pipeline files
- modules_json - Only installed modules found in
modules.json - modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
Run details
- nf-core/tools version 2.13.1
- Run at
2024-04-17 10:26:52
I did manually test the change with the test profile and providing extra_star_align_args = '--outFilterMultimapNmax 50', but it felt excessive to write a separate nf-test only for this one parameter and wasn't sure if I can muddle it through with an existing test without derailing those?
Thank your for doing this, @MatthiasZepper. I think it follows the principle of "least surprise". Don't know how easy it would be, but if you wanted to avoid people inadvertently overwriting preset options, you could test for that and issue a warning. I don't consider it strictly necessary, though -- people who use this option should know what they are doing.
Seems the tests are failing. Before I try to troubleshoot code in a language I do not know... do you, Jonathan, right away know where the problem is?
Sorry, my bad. These should do it.
That looks much better! Thank you very much!
There are two tests failing, but to me, it doesn't seem that this is anyway related to this PR.