json-schema-spec icon indicating copy to clipboard operation
json-schema-spec copied to clipboard

Make evaluation with/against language consistent.

Open handrews opened this issue 4 years ago • 5 comments

We should be consistent when talking about evaluating a schema and an instance together. This is sometimes discussed as being "runtime", which I think is fairly clear although maybe should be called out in the intro material.

Otherwise, I often use "evaluate against" or "validate against", but whether it's "schema evaluated against an instance' or "instance validated against a schema", etc. is pretty random. It probably roughly correlates with whether I was primarily talking about the instance or schema, but maybe not even that.

@ssilverman this is one of several wording issues that could use a meticulous eye. And for that matter, an answer to the question of why it goes one way or the other, even if that's just "we flipped a coin and decided it works this way."

From a user perspective, you're using the schema to evaluate / validate the instance. So the instance is or is not valid (evaluated?) against the schema.

Within the spec, we talk about loading schemas (load time) and evaluating them with/against an instance (runtime). This is more about the implementation process of doing the things the schema keywords indicate.

It's a bit of a vague mess with different bits written from different perspectives, and it's been low-level bugging me for a while.

handrews avatar May 15 '20 05:05 handrews

"evaluation" is more broad. Validation is just one of the things that happen when an instance is evaluated against a schema. Other things can also happen (notably annotation collection, when validation succeeds); also, when validation fails, we don't just get a false result back, but also a collection of errors.

In my in-progress implementation I chose to name the primary API entry point "evaluate", not "validate", because of this. :)

karenetheridge avatar May 15 '20 16:05 karenetheridge

@karenetheridge yeah, the inconsistent terminology is partially due to everything originally being expressed in terms of validation.

handrews avatar May 15 '20 16:05 handrews

"evaluate" would be in line with application of schema to instance where validation isn't the primary concern. I think that's a sort of consensus and enough to enable someone to make a PR for this (or at least part of one as a draft).

Relequestual avatar Jul 30 '20 16:07 Relequestual