Output env "Missing environment variable" if environment variable is set to empty string

[Kibubu]

Bug report

Expected behavior and actual behavior

Expected Empty environment variables are passed as "" into the channel. Maybe a warning is generated.

Actual Variable set to "" within the script will cause "Missing environment variable" error.

Steps to reproduce the problem

process ok{

output:
  env FOO into FOO

  """
    #!/usr/bin/env bash
    
    FOO="bar"
  """
}

process bug1{

output:
  env FOO into FOO1

  """
    #!/usr/bin/env bash
    
    FOO=""
  """
}

process bug2{

output:
  env FOO into FOO2

  """
    #!/usr/bin/env bash
    
    FOO=" "
  """
}

Program output

Output

❯ ./nextflow run bug.nf
N E X T F L O W  ~  version 21.10.6
Launching `bug.nf` [adoring_bardeen] - revision: 5ac926ce0b
executor >  local (3)
executor >  local (3)
[4b/0ef23a] process > ok   [100%] 1 of 1 ✔
[78/50df35] process > bug1 [100%] 1 of 1, failed: 1 ✘
[31/146a5e] process > bug2 [100%] 1 of 1, failed: 1 ✘
Error executing process > 'bug1'

Caused by:
  Missing environment variable: FOO

Command executed:

  #!/usr/bin/env bash
  
  FOO=""

Command exit status:
  0

Command output:
  (empty)

Work dir:
  /home/USER/project/work/78/50df35ea237541b952359785152e2c

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

.nextflow.log

Apr-22 15:40:47.718 [main] DEBUG nextflow.cli.Launcher - $> nextflow run bug.nf
Apr-22 15:40:47.819 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 21.10.6
Apr-22 15:40:47.841 [main] INFO  nextflow.cli.CmdRun - Launching `bug.nf` [adoring_bardeen] - revision: 5ac926ce0b
Apr-22 15:40:47.912 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; plugins-dir=/home/USER/.nextflow/plugins
Apr-22 15:40:47.914 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Apr-22 15:40:47.925 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Apr-22 15:40:47.926 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Apr-22 15:40:47.929 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Apr-22 15:40:47.941 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Apr-22 15:40:47.998 [main] DEBUG nextflow.Session - Session uuid: 11eec7c2-3c1b-43bf-b2de-ec731fd1189a
Apr-22 15:40:47.998 [main] DEBUG nextflow.Session - Run name: adoring_bardeen
Apr-22 15:40:47.999 [main] DEBUG nextflow.Session - Executor pool size: 8
Apr-22 15:40:48.014 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 21.10.6 build 5660
  Created: 21-12-2021 16:55 UTC (17:55 CEST)
  System: Linux 5.15.34-1-MANJARO
  Runtime: Groovy 3.0.9 on OpenJDK 64-Bit Server VM 1.8.0_332-b04
  Encoding: UTF-8 (UTF-8)
  Process: 383398@earth [127.0.1.1]
  CPUs: 8 - Mem: 31.1 GB (13.5 GB) - Swap: 34.2 GB (31.3 GB)
Apr-22 15:40:48.027 [main] DEBUG nextflow.Session - Work-dir: /home/USER/project/work [btrfs]
Apr-22 15:40:48.027 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /home/USER/project/bin
Apr-22 15:40:48.033 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Apr-22 15:40:48.041 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Apr-22 15:40:48.122 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 9; maxThreads: 1000
Apr-22 15:40:48.197 [main] DEBUG nextflow.Session - Session start invoked
Apr-22 15:40:48.692 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Apr-22 15:40:48.781 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-22 15:40:48.781 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-22 15:40:48.784 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Apr-22 15:40:48.788 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=8; memory=31.1 GB; capacity=8; pollInterval=100ms; dumpInterval=5m
Apr-22 15:40:48.802 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 9; maxThreads: 1000
Apr-22 15:40:48.870 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-22 15:40:48.870 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-22 15:40:48.873 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-22 15:40:48.873 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-22 15:40:48.876 [main] DEBUG nextflow.Session - Workflow process names [dsl1]: bug2, bug1, ok
Apr-22 15:40:48.877 [main] DEBUG nextflow.script.ScriptRunner - > Await termination 
Apr-22 15:40:48.877 [main] DEBUG nextflow.Session - Session await
Apr-22 15:40:48.961 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-22 15:40:48.964 [Task submitter] INFO  nextflow.Session - [78/50df35] Submitted process > bug1
Apr-22 15:40:48.969 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-22 15:40:48.970 [Task submitter] INFO  nextflow.Session - [4b/0ef23a] Submitted process > ok
Apr-22 15:40:48.974 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-22 15:40:48.975 [Task submitter] INFO  nextflow.Session - [31/146a5e] Submitted process > bug2
Apr-22 15:40:48.999 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: bug1; status: COMPLETED; exit: 0; error: -; workDir: /home/USER/project/work/78/50df35ea237541b952359785152e2c]
Apr-22 15:40:49.020 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'bug1'

Caused by:
  Missing environment variable: FOO

Command executed:

  #!/usr/bin/env bash
  
  FOO=""

Command exit status:
  0

Command output:
  (empty)

Work dir:
  /home/USER/project/work/78/50df35ea237541b952359785152e2c

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
Apr-22 15:40:49.035 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Missing environment variable: FOO
Apr-22 15:40:49.054 [main] DEBUG nextflow.Session - Session await > all process finished
Apr-22 15:40:49.055 [main] DEBUG nextflow.Session - Session await > all barriers passed
Apr-22 15:40:49.058 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: ok; status: COMPLETED; exit: 0; error: -; workDir: /home/USER/project/work/4b/0ef23aabc59117f9e0c5634ab9ec26]
Apr-22 15:40:49.060 [main] WARN  n.processor.TaskPollingMonitor - Killing pending tasks (1)
Apr-22 15:40:49.063 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: bug2; status: COMPLETED; exit: 0; error: -; workDir: /home/USER/project/work/31/146a5ea11c7d7a0d6ae6af72ab5578]
Apr-22 15:40:49.066 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=1; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=1; retriesCount=0; abortedCount=0; succeedDuration=2ms; failedDuration=5ms; cachedDuration=0ms;loadCpus=1; loadMemory=0; peakRunning=1; peakCpus=1; peakMemory=0; ]
Apr-22 15:40:49.241 [main] DEBUG nextflow.CacheDB - Closing CacheDB done
Apr-22 15:40:49.257 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Environment

• Nextflow version: 21.10.6 build 5660
• Java version: 1.8.0_332
• Operating system: Linux
• Bash version: 5.1.16(1)-release

[Kibubu]

I was unaware of the optional keyword. This might be a solid workaround. But I would have exacted val FOO from FOO.ifEmpty('') to be equivalent to env FOO from FOO.ifEmpty('')

process bug1{
 
output:
  env FOO optional true into FOO
 
  """
    #!/usr/bin/env bash
    
    FOO=""
  """
}
 
 
process bug2{
echo true
input:
  val FOO from FOO.ifEmpty('')
  // env FOO from FOO.ifEmpty('') not working
 
output:
 
  """
    #!/usr/bin/env bash
 
    echo $FOO
  """

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.