Spec ambiguous for struct initialization from read_json() object with extra members

[jlumpe]

My task command outputs a struct which is read from a JSON file using read_json(). The struct has an optional field but I'm encountering an error when the corresponding JSON value is null. This is in version 1.1.3.

A base example which runs as expected is:

version 1.0

struct MyStruct {
	Int x
	String? y
}

task mytask {
	input {
	}

	command <<<
		cat > data.json <<EOL
		{
			"x": 123,
			"y": "foo"
		}
		EOL
	>>>

	output {
		MyStruct data = read_json("data.json")
	}
}

However changing the JSON to the following works with Cromwell but not miniwdl:

{
  "x": 123,
  "y": null
}

Relevant part of error.json:

  "cause": {
    "error": "EvalError",
    "message": "function evaluation failed, 'None' for non-optional input/declaration",
    "node": "decl-data",
    "pos": {
      "source": "/home/jared/projects/gambit/data/misc/211031-apollo_illumina_pe-miniwdl/test/test-null2.wdl",
      "line": 22,
      "column": 19
    },
    "run": "call-test_null",
    "dir": "/home/jared/projects/gambit/data/misc/211031-apollo_illumina_pe-miniwdl/test/20211113_220835_wf/call-test_null"
  },

I also get an error when the JSON object has additional keys which are not in the struct definition:

{
  "x": 123,
  "y": "foo",
  "z": "bar",
}

  "cause": {
    "error": "EvalError",
    "message": "runtime struct initializer doesn't have the required fields with the expected types",
    "node": "decl-data",
    "pos": {
      "source": "/home/jared/projects/gambit/data/misc/211031-apollo_illumina_pe-miniwdl/test/test-extra-key2.wdl",
      "line": 23,
      "column": 19
    },
}

The standard doesn't explicitly state whether this is allowed, (although it is by Cromwell), but I think at the very least the error message is confusing.

[mlin]

@jlumpe miniwdl v1.4.0 fixes the null handling and improves error messages as you suggested. We did not yet change its behavior of rejecting extraneous keys in JSON objects, but tracking that discussion on the OpenWDL forum. Renaming the issue to reflect that. Thanks!

[mlin]

To do: start allowing extra fields, but display warnings during input parsing