nextflow
nextflow copied to clipboard
Sub-workflow is not inheriting replaced param values
Bug report
Expected behavior and actual behavior
When calling .replace on the params object in main.nf, I would expect that any access to the params object in any workflow or sub-workflow would return the updated copy. This is not the case.
Steps to reproduce the problem
.
├── main.nf
├── params.yml
└── sub_main.nf
main.nf
cat main.nf
include { SUB_MAIN } from './sub_main.nf'
params.replace("foo", "bar-has-been-replaced")
workflow MAIN{
take:
numbers
main:
log.error "MAIN::params: ${params}"
SUB_MAIN(numbers)
}
workflow {
main:
log.error "params: ${params}"
MAIN(Channel.of(1, 2, 3))
}
params.yml
foo: "bar"
sub_main.nf
workflow SUB_MAIN{
take:
numbers
main:
log.error "SUB_MAIN::params: ${params}"
numbers.view()
}
Program output
$ nextflow run -params-file params.yml .
N E X T F L O W ~ version 23.10.1
Launching `./main.nf` [astonishing_legentil] DSL2 - revision: a32a34c7c7
ERROR ~ params: [foo:bar-has-been-replaced]
-- Check '.nextflow.log' file for details
ERROR ~ MAIN::params: [foo:bar-has-been-replaced]
-- Check '.nextflow.log' file for details
ERROR ~ SUB_MAIN::params: [foo:bar]
-- Check '.nextflow.log' file for details
1
2
3
Notice how SUB_MAIN::params prints out [foo:bar], not the expected [foo:bar-has-been-replaced].
Environment
- Nextflow version: 23.10.1
- Java version: openjdk version "17.0.3" 2022-04-19 LTS
- Operating system: OSX
- Bash version: zsh 5.9 (x86_64-apple-darwin23.0)
The params map is intended to be immutable, likely this replace will not be supported in the future. The best practice is to only use params in the anonymous workflow and pass params as explicit inputs to subworkflows and processes
I see things like this in nf-core pipelines where params are overwritten (I think).
We are moving away from that pattern. Those params should really just be variables
It would be great to have this be documented somewhere so we don’t have the rug pulled out from us for patterns that are in use today.
We are developing some clearer and stricter guidance on what is allowed in the DSL, since the Groovy compiler allows way more than what is intended for Nextflow. We're still working out some details, but we will definitely provide documentation (and tooling 😄 ) once it's ready.