JSON schema generation incomplete

[jbednar]

At the moment, Param's JSON schema support is meant to work like this:

import param, json, jsonschema

class P(param.Parameterized):
    a = param.Integer(default=5, doc='Int', bounds=(2,30), inclusive_bounds=(True, False))
    b = param.Number(default=4.3, allow_None=True)
    c = param.String(default='foo')

p = P(a=29)
s = p.param.serialize_parameters()
d = json.loads(s)
full_schema = {"type" : "object", "properties" : p.param.schema()}
jsonschema.validate(instance=d, schema=full_schema)

I.e., jsonschema.validate accepts a python dictionary (the output of json.loads(s) on a string s) and a "full" schema (with "object" at the top level). This works, as you can see if you change one of the dictionary entries to something not matching the schema:

d2 = d.copy()
d2.update(dict(a='astring'))

with param.exceptions_summarized():
    jsonschema.validate(instance=d2, schema=full_schema)

ValidationError: 'astring' is not of type 'integer'

However, the validation also "succeeds" in many other cases that one would think would be validation errors:

jsonschema.validate(instance=d2, schema=p.param.schema())
jsonschema.validate(instance=s,  schema=p.param.schema())
jsonschema.validate(instance=42, schema=p.param.schema())
jsonschema.validate(instance="garbage", schema=p.param.schema())

I.e., if you pass not a full schema but the partial schema produced by p.param.schema(), jsonschema.validate will happily accept d2, or s, or pretty much anything you supply (e.g. the string "garbage" or an integer). All of these are rejected correctly if you pass full_schema.

Thus things only work properly when you pass full_schema, and worse, without full_schema they fail silently, which is likely to mislead users into thinking their input is valid when it is not.

Given the awkwardness of having to construct full_schema by hand, and the misleading output when using anything else, one might reasonably ask why p.param.schema() is not simply generating full_schema all the time, both for convenience and to ensure good validation in cases where the programmer is (reasonably) confused.

According to @jlstevens, he was reluctant to generate a full_schema directly because such an object represents a JavaScript "object" type, roughly corresponding to a Python dictionary, whereas here we are representing a Python class instance, which has a dictionary but is not identical to a dictionary. Still, given the clear need to represent full Parameterized objects in JSON (see #520), plus the inconvenience of pasting on the full_schema boilerplate for the schema to be used with jsonschema.validate, plus the highly misleading behavior when supplying anything less than the full_schema, I strongly vote for p.param.schema() to return the equivalent of full_schema directly in all cases.

[jlstevens]

As discussed in #520, the appropriate full_schema is not defined until the conventions needed to represent parameterized objects (including nesting) are decided upon.

I do agree that once we have decided the scheme for #520, we should return full_schema (whatever it is). I think it is unfortunate that validate doesn't raise errors when it should (with the partial schema).