Refering to inputs parameter with valueFrom within step triggers evaluation error

[EricBoix]

Expected Behavior

Assume that in a WorkflowStep you need to refer to a workflow (main) parameter named e.g. main_input. Then the ad-hoc idiom (as documented here) seems to be

inputs:
  main_input: string
steps:
   main_step:
     run: ...
     in:
      sub_input:
        source: main_input  # Ref: https://www.biostars.org/p/378279/
        valueFrom: "$(inputs.main_input)"

Since this idiom seems to be effective in some context, we expect it to handle over the value of main_input parameter to the specified nested Worflow.

Actual Behavior

The above idiom triggers the following error message

ERROR [step main_step] Cannot make job: Expression evaluation error:
Expecting value: line 1 column 1 (char 0)
script was:
01 "use strict";
02 var inputs = {
03     "sub_input": "some-string-from-main-input"
04 };
05 var self = "some-string-from-main-input";
06 var runtime = {
07     "tmpdir": null,
08     "outdir": null
09 };
10 (function(){return ((inputs.main_input));})()
stdout was: 'undefined'
stderr was: ''

Workflow Code

For convenience all the following material is provided by this valuefrom_problem.zip archive: refer to the included Readme.md for instructions.

Consider the echo example of the user_guide, that is relabeled to subworkflow.cwl

#!/usr/bin/env cwl-runner
cwlVersion: v1.1
class: CommandLineTool
baseCommand: echo
inputs:
  sub_input:
    type: string
    inputBinding:
      position: 1
outputs: []

and that, as expected, validates and runs smoothly.

Now consider the following workflow main.cwl

#!/usr/bin/env cwl-runner
  
cwlVersion: v1.0
class: Workflow

requirements:
  SubworkflowFeatureRequirement: {}
  StepInputExpressionRequirement: {}
  InlineJavascriptRequirement: {}

inputs:
  main_input: string

steps:
  main_step:
    run: subworkflow.cwl
    in:
      sub_input: 
        source: main_input  # Ref: https://www.biostars.org/p/378279/
        valueFrom: "$(inputs.main_input)"
    out: []

outputs: []

and the following input file _main-input.yml _

main_input: some-string-from-main-input

Although main.cwl validates (cwl-runner --validate main.cwl) properly when running it one gets

$ cwl-runner main.cwl main-input.yml     
INFO /private/tmp/valuefrom_problem/venv/bin/cwl-runner 1.0.20190915164430
INFO Resolved 'main.cwl' to 'file:///private/tmp/valuefrom_problem/main.cwl'
INFO [workflow ] start
INFO [workflow ] starting step main_step
ERROR [step main_step] Cannot make job: Expression evaluation error:
Expecting value: line 1 column 1 (char 0)
script was:
01 "use strict";
02 var inputs = {
03     "sub_input": "some-string-from-main-input"
04 };
05 var self = "some-string-from-main-input";
06 var runtime = {
07     "tmpdir": null,
08     "outdir": null
09 };
10 (function(){return ((inputs.main_input));})()
stdout was: 'undefined'
stderr was: ''

INFO [workflow ] completed permanentFail
{}
WARNING Final process status is permanentFail
(venv) EBO_PC(eboix): 

Full Traceback

$ cwl-runner --debug main.cwl main-input.yml     
INFO /private/tmp/valuefrom_problem/venv/bin/cwl-runner 1.0.20190915164430
INFO Resolved 'main.cwl' to 'file:///private/tmp/valuefrom_problem/main.cwl'
DEBUG [workflow ] initialized from file:///private/tmp/valuefrom_problem/main.cwl
INFO [workflow ] start
DEBUG [workflow ] {
    "main_input": "some-string-from-main-input"
}
INFO [workflow ] starting step main_step
DEBUG [job step main_step] job input {
    "file:///private/tmp/valuefrom_problem/main.cwl#main_step/sub_input": "some-string-from-main-input"
}
ERROR [step main_step] Cannot make job: Expression evaluation error:
Expecting value: line 1 column 1 (char 0)
script was:
01 "use strict";
02 var inputs = {
03     "sub_input": "some-string-from-main-input"
04 };
05 var self = "some-string-from-main-input";
06 var runtime = {
07     "tmpdir": null,
08     "outdir": null
09 };
10 (function(){return ((inputs.main_input));})()
stdout was: 'undefined'
stderr was: ''

DEBUG 
Traceback (most recent call last):
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/sandboxjs.py", line 378, in execjs
    return cast(JSON, json.loads(stdout))
  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/expression.py", line 304, in do_eval
    strip_whitespace=strip_whitespace)
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/expression.py", line 238, in interpolate
    debug=debug, js_console=js_console)
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/expression.py", line 203, in evaluator
    debug=debug, js_console=js_console)
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/sandboxjs.py", line 382, in execjs
    err, fn_linenum(), stdout, stderr)), err)
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/future/utils/__init__.py", line 400, in raise_from
    exec(execstr, myglobals, mylocals)
  File "<string>", line 1, in <module>
cwltool.sandboxjs.JavascriptException: Expecting value: line 1 column 1 (char 0)
script was:
01 "use strict";
02 var inputs = {
03     "sub_input": "some-string-from-main-input"
04 };
05 var self = "some-string-from-main-input";
06 var runtime = {
07     "tmpdir": null,
08     "outdir": null
09 };
10 (function(){return ((inputs.main_input));})()
stdout was: 'undefined'
stderr was: ''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/workflow.py", line 477, in job
    for newjob in step.iterable:
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/workflow.py", line 399, in try_make_job
    inputobj = postScatterEval(inputobj)
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/workflow.py", line 368, in postScatterEval
    return {k: valueFromFunc(k, v) for k, v in io.items()}
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/workflow.py", line 368, in <dictcomp>
    return {k: valueFromFunc(k, v) for k, v in io.items()}
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/workflow.py", line 365, in valueFromFunc
    timeout=runtimeContext.eval_timeout)
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/cwltool/expression.py", line 307, in do_eval
    raise_from(WorkflowException("Expression evaluation error:\n%s" % Text(e)), e)
  File "/private/tmp/valuefrom_problem/venv/lib/python3.7/site-packages/future/utils/__init__.py", line 400, in raise_from
    exec(execstr, myglobals, mylocals)
  File "<string>", line 1, in <module>
cwltool.errors.WorkflowException: Expression evaluation error:
Expecting value: line 1 column 1 (char 0)
script was:
01 "use strict";
02 var inputs = {
03     "sub_input": "some-string-from-main-input"
04 };
05 var self = "some-string-from-main-input";
06 var runtime = {
07     "tmpdir": null,
08     "outdir": null
09 };
10 (function(){return ((inputs.main_input));})()
stdout was: 'undefined'
stderr was: ''

INFO [workflow ] completed permanentFail
DEBUG [workflow ] {}
{}
WARNING Final process status is permanentFail

Your Environment

• cwltool version: 1.0.20190915164430

Notes

• Notice that the main parameter value is probably properly extracted from the main "source" as illustrated by the line

   03     "sub_input": "some-string-from-main-input"
  

  "Expression evaluation error".
• Could this issue be somehow related with issue #1147 ?
• If the above workflow code were to be correct then even a kludgy workaround (a more "complicated" valueFrom expression?) would be greatly appreciated :-)

[dannmartens]

I have exactly the same issue.

I went to investigate some of the test workflows, specifically https://github.com/common-workflow-language/cwltool/blob/fec7a10466a26e376b14181a88734983cfb1b8cb/cwltool/schemas/v1.1/tests/basename-fields-test.cwl.

    run: echo-file-tool.cwl
    in:
          tool: tool
          in:
              valueFrom: $(inputs.tool.nameroot)
    out: [out]

tool is already declared as inputs at the top level. This works with cwltool, but I have difficulties to understand why it should, for two reasons:

1. echo-file-tool takes only an argument in and another optional name, not tool,
2. the expression seems to imply the existence of inputs at the step level, which is confusing, as the keyword used here is in

The fact that an input argument is named in makes this example very difficult to read.

In my use case, I only need one argument. This trick to rescope tool to the step level forces two arguments down to the step workflow, which then complains it doesn't recognize tool.

[EricBoix]

Thank you for inquiring on this issue @dannmartens . I thus tried fixing the main.cwl provided in this issue (refer above) with the syntax you kindly indicated. If I now consider the steps section as being

steps:
  main_step:
    run: subworkflow.cwl
    in:
      main_input: main_input       # ADDED LINE (where main_input is declared in the inputs section)
      sub_input: 
        # source: main_input         COMMENTED OUT LINE 
        valueFrom: "$(inputs.main_input)"
    out: []

then running with cwltool does not halt with an ERROR message anymore but instead displays the following WARNING:

WARNING Workflow checker warning:
main-fixed.cwl:18:7: 'main_input' is not an input parameter of
                     file:///tmp/valuefrom_problem/subworkflow.cwl,
                     expected sub_input

where line 18 corresponds to the added line main_input: main_input.

I also feel puzzled when trying to understand the underlying implementation logic and particularly when trying to guess why your proposition is indeed effective (up to the warning message). If I were to bet I would go for some (obscure/troubling) "scope importation side effect" of mentioning a reference to a variable (main_input). But even such a guess is not that clear (to me) since the full line has to be main_input: main_input: what are the respective syntactic statuses of the main_input strings when on the left or right side of the semi-colon ? Does main_input: refer to a scope whereas : main_input refers to parameter within that scope ?

And you are right: the general resulting effective syntax is totally confusing (at least for me). Additionally the effectiveness of such a fix being probably based on some underlying implementation detail of cwltool (as opposed to being a documented language feature?) we can neither expect it to work for another implementation nor even to last with cwltool (what happens when cwltool implementation changes : will this "side-effect" still be effective)?

[tetron]

@EricBoix

The inputs object has the inputs for the step, not the inputs for the workflow. The source fields refer to inputs of the workflow. So main_input: main_input means "copy the workflow input main_input to the step input (also called main_input).

The valueFrom fields are evaluated after the the source fields. The inputs object consists of the values copied from source fields.

Does that clear it up?

[dannmartens]

Hi @tetron, thank you for clarifying this.

I modified the test example I mentioned above as follows:

basename-fields-test.cwl

cwlVersion: v1.1
class: Workflow

requirements:
  - class: StepInputExpressionRequirement

inputs:
  tool: File

outputs:
  rootFile:
    type: File
    outputSource: root/out
  extFile:
    type: File
    outputSource: ext/out

steps:
  root:
    run: echo-file-tool.cwl
    in:
      value:
        source: "#tool"
        valueFrom: $(inputs.value.nameroot)
    out: [out]
  ext:
    run: echo-file-tool.cwl
    in:
      value:
        source: "#tool"
        valueFrom: $(inputs.value.nameext)
    out: [out]

echo-file-tool.cwl

cwlVersion: v1.1
class: CommandLineTool
baseCommand: [echo]
inputs:
  value:
    type: string
    inputBinding:
      position: 1
stdout: out.txt
outputs:
  out:
    type: stdout

Now there is no complaint about the rescoped value; before the change we got:

WARNING Workflow checker warning: basename-fields-test.cwl:22:7: 'tool' is not an input parameter of file:///echo-file-tool.cwl, expected value

Still, one odd thing remaining is that I don't get the warning anymore when I intentionally try to break it again:

  ext:
    run: echo-file-tool.cwl
    in:
      value:
        source: "#tool"
        valueFrom: $(inputs.value.nameext)
      unknownParameter:
        valueFrom: "sdfdsfds"
    out: [out]

Now unknownParameter is ignored.

[BJWiley233]

How do you source 2 things from workflow such as if you want to create a string from 2 strings?