Add support for JSON Schema validation assert?

[NateEag]

I often use JSON Schema for specifying the exact structure of responses from an API.

I would love it if Hurl had a way to assert that a response validates against a given JSON Schema.

[NateEag]

One way to support checking JSON Schemas against response bodies would be to add a command assert that requires an external command to be specified (presumably with support for capture substitutions in the command, maybe with a default of passing the response body to the command on standard input if no captures are supplied?).

When the assert is checked, it runs the specified command. An exit status of 0 means the assert succeeds, while a non-zero exit status means it failed.

That should make it straightforward for people who want JSON Schema validations to do it themselves using their tool of choice.

It would also make it simple for people to add other arbitrary other assertions they want to their Hurl tests, like maybe using jp to do checks using JMESPath.

Would the maintainers be open to adding something like this?

[fabricereix]

Hi @NateEag, Testing the response should probably fit into the Hurl Assert model (query + response).

Let's have have concrete examples to discuss about it.

For the following JSON response

{
  "productId": 1,
  "productName": "A green door",
  "price": 12.50,
  "tags": [ "home", "green" ]
}

and the following schema file schema.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://example.com/product.schema.json",
  "title": "Product",
  "description": "A product in the catalog",
  "type": "object"
}

We could have something like

[Asserts]
body startsWith "{"
body validateJSONSchema file,schema.json;
body jsonSchema file,schema.json;
body isJSONSchemaValid file,schema.json;
body isValidatedBy file,schema.json;
body isValidatedByJSONSchema file,schema.json;

It reuses the following constructs already existing in Hurl:

• query body that returns the body as a string.
• file,...; that retrieves a content from an external file

validateJSONSchema/jsonSchema would be a new predicate.

What do you think?

[Applequist]

Hi, Any news on this feature? Is it considered for a not so remote next release :) ?

[Krzysztow]

Based on the second part of the response in https://github.com/Orange-OpenSource/hurl/issues/1619#issuecomment-1582745670, is this issue considered to have green light for development and potential PRs?

[fabricereix]

yes @Krzysztow, you could start working on it. we need to decide which predicate name to choose though. I like body jsonSchema file,schema.json; This might not be very explicit in itself, but within an [Asserts] section, I find it clear enough. what do you think @lepapareil @jcamiel ?

[fabricereix]

it would be good to support also embedded schema.

body jsonSchema ```{ "type": "string" }```

That's why I think we need to include jsonSchema in the name.

[lepapareil]

As all predicates are verbs, perhaps we could re-use matches :

[Asserts]
body matchesJsonSchema file,schema.json;

[Asserts]
body matchesXmlSchema file,schema.xsd;