process.ext set with withName overrides configuration of all processes

[grst]

Bug report

Expected behavior and actual behavior

• Expected: Options set with withName only affect the corresponding process(es)
• Actual: If process.ext gets initalized for all processes, updating it for a single process later overrides that value in all processes

Steps to reproduce the problem

Execute

nextflow run main.nf 

with the following three files

main.nf

#!/usr/bin/env nextflow

nextflow.enable.dsl = 2

include { FOO; FOO as BAR } from "./module.nf"

workflow {
    FOO()
    BAR()
}

nextflow.config

process {
    ext.args = "Hello World!"
    withName: 'BAR' {
        ext.args   = "Hi World!" 
	 cpus = 1
    }
}

module.nf

process FOO {
    exec:
    println task.ext.args
}

Program output

N E X T F L O W  ~  version 21.04.0
Launching `main.nf` [hungry_lalande] - revision: 352bf9af95
executor >  local (2)
[6c/88b621] process > FOO [100%] 1 of 1 ✔
[f3/2daf6d] process > BAR [100%] 1 of 1 ✔
Hi World!
Hi World!

Environment

• Nextflow version: Tested with 21.04.0 and 21.10.0-SNAPSHOT
• Java version: openjdk 11.0.8-internal 2020-07-14
• Operating system: linux
• Bash version: GNU bash, version 5.1.8(1)-release (x86_64-pc-linux-gnu)

Additional context

This bug was discovering while experimenting with the new nf-core dsl2.0 template together with @mahesh-panchal and @drpatelh

[grst]

Interestingly, with the following nextflow.config, it works as expected:

process.ext.args = "Hello World!"

process {
    withName: 'BAR' {
        ext.args   = "Hi World!" 
	cpus = 1
    }
}

N E X T F L O W  ~  version 21.04.0
Launching `main.nf` [sleepy_brenner] - revision: 352bf9af95
executor >  local (2)
[08/1de118] process > FOO [100%] 1 of 1 ✔
[92/af5129] process > BAR [100%] 1 of 1 ✔
Hi World!
Hello World!

[pditommaso]

In my test, the patch for #2382 solves also this issue.

[mahesh-panchal]

The patch doesn't work for this issue however for me.

$ NXF_VER=21.10.0-SNAPSHOT CAPSULE_RESET=true nextflow info
CAPSULE: Downloading dependency io.nextflow:nf-httpfs:jar:21.10.0-20211101.112009-4
CAPSULE: Downloading dependency io.nextflow:nextflow:jar:21.10.0-20211101.112009-4
CAPSULE: Downloading dependency io.nextflow:nf-commons:jar:21.10.0-20211101.112009-4
  Version: 21.10.0-SNAPSHOT build 5639
  Created: 01-11-2021 11:20 UTC (12:20 CEST)
  System: Mac OS X 10.16
  Runtime: Groovy 3.0.9 on OpenJDK 64-Bit Server VM 11.0.9.1+1-LTS
  Encoding: UTF-8 (UTF-8)

$ NXF_VER=21.10.0-SNAPSHOT nextflow run main.nf
N E X T F L O W  ~  version 21.10.0-SNAPSHOT
Launching `main.nf` [gigantic_ekeblad] - revision: 02be96b9e9
executor >  local (2)
[4c/d98e9d] process > FOO [100%] 1 of 1 ✔
[d4/ffe2a6] process > BAR [100%] 1 of 1 ✔
Hi World!
Hi World!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[drpatelh]

Bump. Might be worth giving it another shot to see if it works again with the latest edge release @grst @mahesh-panchal

[pditommaso]

It does not, currently this notation works

 process {
      ext { args = "Hello World!" } 
      cpus = 1 
      withName:BAR {
          ext { args = "Ciao mondo!" } 
          cpus = 2
      }
  }

However, this does not

  process {
      ext.args = "Hello World!" 
      cpus = 1 
      withName:BAR {
          ext.args = "Ciao mondo!"
          cpus = 2
      }
  }

I've isolated the problem into this test

[jorgeaguileraseqera]

I think the problem comes at

https://github.com/nextflow-io/nextflow/blob/master/modules/nextflow/src/main/groovy/nextflow/config/ConfigParser.groovy#L335-L341

because we are not taking into account when we are in a nested ConfigObject (stored in profileStack)

I'll take a look to it

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.