Investigate OpenMINDS as an option for ngff validation

[will-moore]

See Intro video at https://vimeo.com/605602233 (start at 9 minutes)

It seems that OpenMINDS has a way of describing schemas, and tools that can convert those schemas into JSON-ld, html doc, and tools that can use the schema to provide APIs for creating data records of schema Objects.

Testing some code, following example at https://github.com/HumanBrainProject/openMINDS_generator

// This step clones the schemas into default location of ~/.openMINDS_python/
openMINDS.version_manager.init()

# initiate the helper class for the dynamic usage of a specific openMINDS version
helper = openMINDS.Helper()
mycollection = helper.create_collection()

# create a metadata instance for (e.g.) the openMINDS Person schema
# the `add_core_person()` is auto-generated from the "core" schema, actors/person.schema.json
# The givenName is specified as 'required' so this will fail if you don't supply at least 1 argument
person_open = mycollection.add_core_person(givenName="open")

# test another object
mycollection.add_SANDS_coordinatePoint()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: add_SANDS_coordinatePoint() missing 2 required positional arguments: 'coordinateSpace' and 'coordinates'

# however, there doesn't seem to be much validation. This doesn't fail
point = mycollection.add_SANDS_coordinatePoint("test", "fail")
mycollection.save("./myInvalidTestCollection/")

That created `./myInvalidCollection/coordinatePoint/d734b6d4-415c-11ec-a0c9-a45e60c9f76d.jsonld

{
    "@id": "https://localhost/coordinatePoint/d734b6d4-415c-11ec-a0c9-a45e60c9f76d",
    "@type": "https://openminds.ebrains.eu/SANDS/Coordinatepoint",
    "coordinateSpace": "test",
    "coordinates": "fail",
    "@context": {
        "@vocab": "https://openminds.ebrains.eu/vocab/"
    }
}

which appears invalid looking at the schema https://github.com/HumanBrainProject/openMINDS_SANDS/blob/v3/schemas/miscellaneous/coordinatePoint.schema.tpl.json

There is a validator but it only checks that the file is JSON: json.loads(f.read())

$ cd myInvalidTestCollection/
$ python /Users/wmoore/Desktop/HumanBrainProject/openMINDS_generator/validator/schema_validator.py
person/46a6bd9e-4155-11ec-a0c9-a45e60c9f76d.jsonld: PASSED
contactInformation/63ae260c-4155-11ec-a0c9-a45e60c9f76d.jsonld: PASSED
coordinatePoint/d734b6d4-415c-11ec-a0c9-a45e60c9f76d.jsonld: PASSED

cc @joshmoore @jburel

[will-moore]

According to the linked issue above, openMINDS uses JSON-Schema tools for validation. Testing on-line with https://www.jsonschemavalidator.net/ OR with python: https://pypi.org/project/jsonschema/, using the following schema, simplified from openMINDS original....

{
  "$id": "https://openminds.ebrains.eu/core/person?format=json-schema",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "_categories": [
    "legalPerson"
  ],
  "_type": "https://openminds.ebrains.eu/core/Person",
  "description": "Structured information on a person.",
  "properties": {
    "@id": {
      "description": "Metadata node identifier.",
      "type": "string"
    },
    "@type": {
      "const": "https://openminds.ebrains.eu/core/Person",
      "type": "string"
    },
    "https://openminds.ebrains.eu/vocab/givenName": {
      "_instruction": "Enter the given name of this person.",
      "description": "Name given to a person, including all potential middle names, but excluding the family name.",
      "title": "givenName",
      "type": "string"
    }
  },
  "required": [
    "@id",
    "@type",
    "givenName"
  ],
  "type": "object"
}

The following JSON passes validation, even though the givenName is an object not a string. Extra fields are also ignored (e.g. foo).

{
    "@id": "https://localhost/person/46a6bd9e-4155-11ec-a0c9-a45e60c9f76d",
    "@type": "https://openminds.ebrains.eu/core/Person",
    "familyName": {"test": true},
    "givenName":
      {"test": true, "@type": "whatever"},
    "foo": {"test": false},
    "@context": {
        "@vocab": "https://openminds.ebrains.eu/vocab/"
    }
}

EDIT: If I replace https://openminds.ebrains.eu/vocab/givenName with simply givenName then the type of givenName is enforced.

[joshmoore]

Do you have a link to the json-schema that was used for validation?

[will-moore]

The schema and the JSON are both in that comment. Text is a bit jumbled - I'll fix...

[will-moore]

I guess the fact that validation doesn't fail for invalid types when the properties key is a URL is what's meant by "The validation of JSON-LDs across links definitely more complicated." https://github.com/HumanBrainProject/openMINDS_core/issues/256#issuecomment-966415663