null values in json result in json schema validation error

[wdduncan]

We have dataset that is submitted as json, but it contains null values. E.g.:

{
        "bin_name": "bins.1",
        "gtdbtk_domain": null,
 }

In the yaml, we define the bin object like this:

MAG bin:
    attributes:
      bin name:
        range: string
     gtdbtk domain:
        required: false
        range: string

When we run the validation, we receive the error:

None: None is not of type 'string'

In the generated json schema, MAG bin is specified like so:

"MAGBin": {
         "additionalProperties": false,
         "description": "",
         "properties": {
            "bin_name": {
               "type": "string"
          },
          "gtdbtk_domain": {
               "type": "string"
            }
  }

How can we get it also accept null when validating?

This discussion may be relevant: https://groups.google.com/g/jsonschema/c/mD6GDca4zN8?pli=1

cc @cmungall

[cmungall]

@wdduncan is this still an issue?

how are you validating? can you provide a test case?

[wdduncan]

This is an issue with jsonschema and not with linkml per se. In the below example, the email field is not required, but an violation is thrown. Do we want to add functionality to linkml to address this? Perhaps it might be a nice feature to have linkml filter out null/None?

script:

from jsonschema import validate

schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "email": {"type": "string"},
    },
    "required": ["name"],
}

data1 = {"name": "foo", "email": "[email protected]"}
validate(data1, schema)  # passes

data2 = {"name": "foo"}
validate(data2, schema)  # passes

data3 = {"name": "foo", "email": None} 
validate(data3, schema)  # FAILS

returns:

Failed validating 'type' in schema['properties']['email']:
    {'type': 'string'}

On instance['email']:
    None

[gaurav]

I think you can fix this by defining the "type" as ["string", "null"], according to this StackOverflow answer. I'm not sure what the best way of incorporating this into LinkML would be.

[wdduncan]

Thanks @gaurav ! I was totally unaware of this :)

I have confirmed that modifying the example schema like so works.

schema = {
  "type": "object",
  "properties": {
    "name": {"type": "string"},
    "email": {"type": ["string", "null"]}  # <---- add "null"
  },
  "required": ["name"],
}

data3 = {"name": "foo", "email": None} now passes validation.

[cmungall]

I suggest both a flag at the generator level, and an extension at the metamodel. Let's restrict this ticket to the generator level change and make a new ticket for metamodel extensions

On Wed, Oct 13, 2021 at 7:35 AM Bill Duncan @.***> wrote:

Thanks @gaurav https://github.com/gaurav ! I was totally unaware of this :)

I have confirmed that modifying the example schema like so works.

schema = { "type": "object", "properties": { "name": {"type": "string"}, "email": {"type": ["string", "null"]} # <---- add "null" }, "required": ["name"], }

data3 = {"name": "foo", "email": None} now passes validation.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/linkml/linkml/issues/102#issuecomment-942370618, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAMMOLIAFWA7OWD36WNDMLUGWKJ7ANCNFSM4Z2LTZEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[wdduncan]

@cmungall This issue looks stale. Perhaps we should close it or assign it to someone else?