nextflow
nextflow copied to clipboard
Output env "Missing environment variable" if environment variable is set to empty string
Bug report
Expected behavior and actual behavior
Expected
Empty environment variables are passed as ""
into the channel. Maybe a warning is generated.
Actual Variable set to "" within the script will cause "Missing environment variable" error.
Steps to reproduce the problem
process ok{
output:
env FOO into FOO
"""
#!/usr/bin/env bash
FOO="bar"
"""
}
process bug1{
output:
env FOO into FOO1
"""
#!/usr/bin/env bash
FOO=""
"""
}
process bug2{
output:
env FOO into FOO2
"""
#!/usr/bin/env bash
FOO=" "
"""
}
Program output
Output
❯ ./nextflow run bug.nf
N E X T F L O W ~ version 21.10.6
Launching `bug.nf` [adoring_bardeen] - revision: 5ac926ce0b
executor > local (3)
executor > local (3)
[4b/0ef23a] process > ok [100%] 1 of 1 ✔
[78/50df35] process > bug1 [100%] 1 of 1, failed: 1 ✘
[31/146a5e] process > bug2 [100%] 1 of 1, failed: 1 ✘
Error executing process > 'bug1'
Caused by:
Missing environment variable: FOO
Command executed:
#!/usr/bin/env bash
FOO=""
Command exit status:
0
Command output:
(empty)
Work dir:
/home/USER/project/work/78/50df35ea237541b952359785152e2c
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
.nextflow.log
Apr-22 15:40:47.718 [main] DEBUG nextflow.cli.Launcher - $> nextflow run bug.nf
Apr-22 15:40:47.819 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 21.10.6
Apr-22 15:40:47.841 [main] INFO nextflow.cli.CmdRun - Launching `bug.nf` [adoring_bardeen] - revision: 5ac926ce0b
Apr-22 15:40:47.912 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; plugins-dir=/home/USER/.nextflow/plugins
Apr-22 15:40:47.914 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Apr-22 15:40:47.925 [main] INFO org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Apr-22 15:40:47.926 [main] INFO org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Apr-22 15:40:47.929 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Apr-22 15:40:47.941 [main] INFO org.pf4j.AbstractPluginManager - No plugins
Apr-22 15:40:47.998 [main] DEBUG nextflow.Session - Session uuid: 11eec7c2-3c1b-43bf-b2de-ec731fd1189a
Apr-22 15:40:47.998 [main] DEBUG nextflow.Session - Run name: adoring_bardeen
Apr-22 15:40:47.999 [main] DEBUG nextflow.Session - Executor pool size: 8
Apr-22 15:40:48.014 [main] DEBUG nextflow.cli.CmdRun -
Version: 21.10.6 build 5660
Created: 21-12-2021 16:55 UTC (17:55 CEST)
System: Linux 5.15.34-1-MANJARO
Runtime: Groovy 3.0.9 on OpenJDK 64-Bit Server VM 1.8.0_332-b04
Encoding: UTF-8 (UTF-8)
Process: 383398@earth [127.0.1.1]
CPUs: 8 - Mem: 31.1 GB (13.5 GB) - Swap: 34.2 GB (31.3 GB)
Apr-22 15:40:48.027 [main] DEBUG nextflow.Session - Work-dir: /home/USER/project/work [btrfs]
Apr-22 15:40:48.027 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /home/USER/project/bin
Apr-22 15:40:48.033 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Apr-22 15:40:48.041 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Apr-22 15:40:48.122 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 9; maxThreads: 1000
Apr-22 15:40:48.197 [main] DEBUG nextflow.Session - Session start invoked
Apr-22 15:40:48.692 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Apr-22 15:40:48.781 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-22 15:40:48.781 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-22 15:40:48.784 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Apr-22 15:40:48.788 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=8; memory=31.1 GB; capacity=8; pollInterval=100ms; dumpInterval=5m
Apr-22 15:40:48.802 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 9; maxThreads: 1000
Apr-22 15:40:48.870 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-22 15:40:48.870 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-22 15:40:48.873 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Apr-22 15:40:48.873 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Apr-22 15:40:48.876 [main] DEBUG nextflow.Session - Workflow process names [dsl1]: bug2, bug1, ok
Apr-22 15:40:48.877 [main] DEBUG nextflow.script.ScriptRunner - > Await termination
Apr-22 15:40:48.877 [main] DEBUG nextflow.Session - Session await
Apr-22 15:40:48.961 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-22 15:40:48.964 [Task submitter] INFO nextflow.Session - [78/50df35] Submitted process > bug1
Apr-22 15:40:48.969 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-22 15:40:48.970 [Task submitter] INFO nextflow.Session - [4b/0ef23a] Submitted process > ok
Apr-22 15:40:48.974 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Apr-22 15:40:48.975 [Task submitter] INFO nextflow.Session - [31/146a5e] Submitted process > bug2
Apr-22 15:40:48.999 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: bug1; status: COMPLETED; exit: 0; error: -; workDir: /home/USER/project/work/78/50df35ea237541b952359785152e2c]
Apr-22 15:40:49.020 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'bug1'
Caused by:
Missing environment variable: FOO
Command executed:
#!/usr/bin/env bash
FOO=""
Command exit status:
0
Command output:
(empty)
Work dir:
/home/USER/project/work/78/50df35ea237541b952359785152e2c
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
Apr-22 15:40:49.035 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Missing environment variable: FOO
Apr-22 15:40:49.054 [main] DEBUG nextflow.Session - Session await > all process finished
Apr-22 15:40:49.055 [main] DEBUG nextflow.Session - Session await > all barriers passed
Apr-22 15:40:49.058 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: ok; status: COMPLETED; exit: 0; error: -; workDir: /home/USER/project/work/4b/0ef23aabc59117f9e0c5634ab9ec26]
Apr-22 15:40:49.060 [main] WARN n.processor.TaskPollingMonitor - Killing pending tasks (1)
Apr-22 15:40:49.063 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: bug2; status: COMPLETED; exit: 0; error: -; workDir: /home/USER/project/work/31/146a5ea11c7d7a0d6ae6af72ab5578]
Apr-22 15:40:49.066 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=1; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=1; retriesCount=0; abortedCount=0; succeedDuration=2ms; failedDuration=5ms; cachedDuration=0ms;loadCpus=1; loadMemory=0; peakRunning=1; peakCpus=1; peakMemory=0; ]
Apr-22 15:40:49.241 [main] DEBUG nextflow.CacheDB - Closing CacheDB done
Apr-22 15:40:49.257 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye
Environment
- Nextflow version: 21.10.6 build 5660
- Java version: 1.8.0_332
- Operating system: Linux
- Bash version: 5.1.16(1)-release
I was unaware of the optional
keyword. This might be a solid workaround.
But I would have exacted val FOO from FOO.ifEmpty('')
to be equivalent to env FOO from FOO.ifEmpty('')
process bug1{
output:
env FOO optional true into FOO
"""
#!/usr/bin/env bash
FOO=""
"""
}
process bug2{
echo true
input:
val FOO from FOO.ifEmpty('')
// env FOO from FOO.ifEmpty('') not working
output:
"""
#!/usr/bin/env bash
echo $FOO
"""
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.