nextflow
nextflow copied to clipboard
Order of 'includeConfig' and 'profiles' in nextflow.config seems to affect nested property overrides
Bug report
Expected behavior and actual behavior
Hi Nextflow team,
I expected that the order in which 'includeConfig' and 'profiles' are declared in nextflow.config would have no effect on property values. However, I think I'm seeing that 'profiles' must be declared after any 'includeConfig' statements, in order for profile overrides to work correctly. In particular, this seems to affect what I'd describe as nested properties.
Steps to reproduce the problem
Given the following two files:
nextflow.config:
profiles {
hpc {
process {
process_nested.value = "hpc.config"
process_nonest = "hpc.config"
withLabel: "labelExample" {
memory = 400
}
}
}
}
includeConfig "memory.config"
and memory.config:
process {
process_nested.value = "memory.config"
process_nonest = "memory.config"
withLabel: "labelExample" {
memory = 4
}
}
and using the following command line invocation:
nextflow config -profile hpc
The nested properties 'process_nested.value' and 'labelExample' fail to be overridden by the hpc profile, while 'process_nonest' is correctly overridden.
(Of note, the problematic overrides work as expected once the 'includeConfig' declaration is moved above the 'profiles' declaration)
Program output
the unexpected output is as follows:
process {
process_nested {
value = 'memory.config'
}
process_nonest = 'hpc.config'
withLabel:labelExample {
memory = 4
}
}
(No .nextflow.log is generated by nextflow config)
Environment
- Nextflow version: Both 23.10.1 and 23.04.4
- Java version: Java/17.0.4
- Operating system: Linux
- Bash version: GNU bash, version 4.2.46(2)-release
Additional context
The example provided is for the nextflow config command, but I'm also seeing similar behavior for nextflow run