nextflow icon indicating copy to clipboard operation
nextflow copied to clipboard

Sub-workflow is not inheriting replaced param values

Open nh13 opened this issue 1 year ago • 5 comments
trafficstars

Bug report

Expected behavior and actual behavior

When calling .replace on the params object in main.nf, I would expect that any access to the params object in any workflow or sub-workflow would return the updated copy. This is not the case.

Steps to reproduce the problem

.
├── main.nf
├── params.yml
└── sub_main.nf
main.nf
cat main.nf 
include { SUB_MAIN } from './sub_main.nf'

params.replace("foo", "bar-has-been-replaced")

workflow MAIN{
	take:
	numbers
	
	main:
	log.error "MAIN::params: ${params}"
	SUB_MAIN(numbers)
}

workflow {
	main:
	log.error "params: ${params}"
	MAIN(Channel.of(1, 2, 3))
}
params.yml
foo: "bar"
sub_main.nf
workflow SUB_MAIN{
	take:
	numbers

	main:
	log.error "SUB_MAIN::params: ${params}"
	numbers.view()
}

Program output

$ nextflow run -params-file params.yml .
N E X T F L O W  ~  version 23.10.1
Launching `./main.nf` [astonishing_legentil] DSL2 - revision: a32a34c7c7
ERROR ~ params: [foo:bar-has-been-replaced]

 -- Check '.nextflow.log' file for details
ERROR ~ MAIN::params: [foo:bar-has-been-replaced]

 -- Check '.nextflow.log' file for details
ERROR ~ SUB_MAIN::params: [foo:bar]

 -- Check '.nextflow.log' file for details
1
2
3

Notice how SUB_MAIN::params prints out [foo:bar], not the expected [foo:bar-has-been-replaced].

Environment

  • Nextflow version: 23.10.1
  • Java version: openjdk version "17.0.3" 2022-04-19 LTS
  • Operating system: OSX
  • Bash version: zsh 5.9 (x86_64-apple-darwin23.0)

nh13 avatar May 09 '24 06:05 nh13

The params map is intended to be immutable, likely this replace will not be supported in the future. The best practice is to only use params in the anonymous workflow and pass params as explicit inputs to subworkflows and processes

bentsherman avatar May 09 '24 15:05 bentsherman

I see things like this in nf-core pipelines where params are overwritten (I think).

nh13 avatar May 10 '24 22:05 nh13

We are moving away from that pattern. Those params should really just be variables

bentsherman avatar May 11 '24 19:05 bentsherman

It would be great to have this be documented somewhere so we don’t have the rug pulled out from us for patterns that are in use today.

nh13 avatar May 12 '24 02:05 nh13

We are developing some clearer and stricter guidance on what is allowed in the DSL, since the Groovy compiler allows way more than what is intended for Nextflow. We're still working out some details, but we will definitely provide documentation (and tooling 😄 ) once it's ready.

bentsherman avatar May 13 '24 14:05 bentsherman