cwltool throws validation errors despite consistent type declarations

[adrabent]

---

Expected Behavior

cwltool --validate should not show any warnings

Actual Behavior

cwltool throws a workflow checker warning, although all types are chosen consistently. The warning output complain about "incompatibility" although both source and sink are of the same type.

Workflow Code

minimal_workflow.cwl:

class: Workflow
cwlVersion: v1.2
id: minimal_example
label: minimal_example
inputs:
  - id: msin
    type: File
outputs:
  - id: msout
    outputSource:
        - subworkflow_1/msout
        - subworkflow_2/msout
    type: File[]?
    linkMerge: merge_flattened
steps:
  - id: subworkflow_1
    in:
      - id: msin
        source: msin
    out:
      - id: msout
    run: subworkflow.cwl
  - id: subworkflow_2
    in:
      - id: msin
        source: msin
    out:
      - id: msout
    run: subworkflow.cwl
requirements:
  - class: SubworkflowFeatureRequirement

subworkflow.cwl:

class: Workflow
cwlVersion: v1.2
id: subworkflow
label: subworkflow
inputs:
  - id: msin
    type: File
outputs:
  - id: msout
    outputSource:
        - msin
    type: File[]?
    linkMerge: merge_nested
steps:
  - id: pass
    in:
      - id: msin
        source: msin
    out:
      - id: msout
    run: pass.cwl

pass.cwl ("do nothing"):

class: CommandLineTool
cwlVersion: v1.2
id: pass
baseCommand: echo
inputs:
  - id: msin
    type: File
outputs:
  - id: msout
    type: File?
    outputBinding:
      outputEval: $(inputs.msin)
requirements:
  - class: DockerRequirement
    dockerPull: ubuntu:22.04

I call the command like this:

cwltool --validate minimal_workflow.cwl

Full Traceback

INFO /usr/local/conda/envs/cwl/bin/cwltool 3.1.20250110105449
INFO Resolved 'minimal_workflow.cwl' to 'file:///home/alex/debug/minimal_workflow.cwl'
WARNING Workflow checker warning:
minimal_workflow.cwl:21:9: Source 'msout' of type ["null", {"type": "array", "items": "File"}] may
                           be incompatible
minimal_workflow.cwl:13:5:   with sink 'msout' of type ["null", {"type": "array", "items":
                             "File"}]
minimal_workflow.cwl:9:5:    source has linkMerge method merge_flattened
minimal_workflow.cwl:28:9: Source 'msout' of type ["null", {"type": "array", "items": "File"}] may
                           be incompatible
minimal_workflow.cwl:13:5:   with sink 'msout' of type ["null", {"type": "array", "items":
                             "File"}]
minimal_workflow.cwl:9:5:    source has linkMerge method merge_flattened
minimal_workflow.cwl is valid CWL.

Your Environment

• cwltool version: 3.1.20250110105449

What works

• Change all types to non-null types: File? --> File and File[]? --> File[]

[lonbar]

The workflow as given fails for me because of a missing MultipleInputFeatureRequirement, but I can reproduce it once I have added this. Looking at the workflow there does appear to be a potential type mismatch, but this isn't displayed by the warning correctly.

subworkflow.cwl has an output of File[]?, which ultimately is a shorthand for ["null", {"type": "array", "items": "File"}] (i.e. it is either null, or File[] (an array of Files). Several of these outputs are concatenated in minimal_workflow.cwl, which means that you have an "array of optional Files", not an "optional array of Files". minimal_workflow.cwl expects the type of msout to be the latter.

Specifically (if I'm understanding this correctly), the [] and ? need to be switched. You can do this by expanding the type of msout in minimal_workflow.cwl as follows:

outputs:
  - id: msout
    outputSource:
        - subworkflow_1/msout
        - subworkflow_2/msout
    type: 
        type: array
        items: ["null", File]
    linkMerge: merge_flattened

With this change the warning disappears for me with the given version of cwltool:

$ cat minimal_workflow.cwl && cwltool --validate minimal_workflow.cwl 
class: Workflow
cwlVersion: v1.2
id: minimal_example
label: minimal_example
inputs:
  - id: msin
    type: File
outputs:
  - id: msout
    outputSource:
        - subworkflow_1/msout
        - subworkflow_2/msout
    type: 
        type: array
        items: ["null", File]
    linkMerge: merge_flattened
steps:
  - id: subworkflow_1
    in:
      - id: msin
        source: msin
    out:
      - id: msout
    run: subworkflow.cwl
  - id: subworkflow_2
    in:
      - id: msin
        source: msin
    out:
      - id: msout
    run: subworkflow.cwl
requirements:
  - class: SubworkflowFeatureRequirement
  - class: MultipleInputFeatureRequirement
INFO /home/matthijs/.local/bin/cwltool 3.1.20250110105449
INFO Resolved 'minimal_workflow.cwl' to 'file:///home/matthijs/minimal_workflow.cwl'
minimal_workflow.cwl is valid CWL.
$

The issue therefore really seems to be that cwltool isn't displaying the type of the sink correctly in the warning.