Semantic JSON assertion

[marcodpt]

Problem to solve

Firstly, I would like to thank this wonderful project and the efforts of its developers in the open source community.

I consider HURL excellent for testing Rest API.

However, the tests themselves could function as documentation if written well.

One of the things that prevents me from doing this is being forced to put unformatted JSON in the tests.

Another problem that it generates is that any ordering change in the properties and we have to correct all the tests.

These seem like simple problems, but they become serious and impeding as the number of tests increases and the complexity of the JSON API data increases.

This problem has already been mentioned and discussed: #272 #270

Proposal

My proposal is to ignore the problem of semantic validation of XML for now and start by solving the problem of semantic validation of JSON.

Libraries like serde json create the Value structure that allows you to compare JSON semantically.

Multiline body starting with ```json should be parsed with the library and the server response as well.

assert_eq!(
    test_multi_line_json.parse::<serde_json::Value>().unwrap(),
    server_response.parse::<serde_json::Value>().unwrap()
);

Additional context and resources

serde_json

JSON semantic compare in rust

Tasks to complete

• [ ] parse with serde json the expected multiline JSON defined in tests.hurl
• [ ] parse the response body
• [ ] in case of any error of parsing the test fail
• [ ] compare both Value with the simple eq operator in rust

[jcamiel]

Hi @marcodpt

We're currently preparing the work for better error reporting and this work is the route to semantic difference. A first batch of refacto is to be able to able to show better diff with multiline ```

The multiline check will not be semantic, whereas the JSON (and XML) body will be semantic. This way, user could choose to have semantic or "raw" diff.

To be clear (unless I'm mistaken of what we want @fabricereix )

Not semantic, raw diff, whitespaces sensitive:

GET https://foo.com/json
HTTP 200
```json
{ 
  "bar": true
}
```

GET https://foo.com/csv
HTTP 200
```
aaa,bbb,ccc
aaa,bbb,ccc
```

=> current work is to improve the error diag

Semantic, whitespaces unsensitive:

GET https://foo.com/json
HTTP 200
{ 
  "bar": true
}

[fabricereix]

This one is clearly not semantic

```
{ 
  "bar": true
}
```

but I'm not sure when the string type json is specified.

```json
{ 
  "bar": true
}
```

Setting such a request body adds automatically the `Content-Type: application/json' header. So maybe this one should also be semantic.

[jcamiel]

Yes it makes sens

[marcodpt]

I would like to thank you very much for your feedback.

I wasn't trying to argue about whether ```json should be semantic or not.

Actually, to be honest, I had forgotten that sending a direct json without a backtick works.

What I really miss is JSON semantic validation, which will allow me to migrate tests and documentation to the .hurl file.

I don't work much with XML in rust, but JSON seems like a not very difficult problem, but I don't really understand anything about the internal workings of hurl.

Semantic validation, for example, allows you to define the specification of an API in a .hurl file and at the same time serve as tests.

That in itself is beautiful, in the file format you define.

[marcodpt]

I work a lot writing servers using Rest API and servers for communicating with IOT devices.

I consider that .hurl files would be perfect for defining an API specification.

But without semantic validation the work becomes much more difficult, and the idea of ​​freezing a specification and building servers is a little weakened if I constantly have to change the .hurl files due to a change in order in the properties or if the files are unreadable by human beings.

[fabricereix]

We could use the same output as jd