Add JSON content validation

[eric91]

I'm starting using rl_json a lot and find it quiet useful for Tcl, as it allows typed string representation to Tcl easily and efficiently. However, I have now a lot of code to validate content of JSON (array, object content, field type, ...). Could an easy way to check / scan JSON be added? Something similar to schema validation function in tDOM? Or based on JSON tempate?

[cyanogilvie]

I built something like this into rltest: rltest compare_json. I clearly haven't done the work to properly finish rltest as an opensource project (like, say, documenting it), but it's there and works quite well for that. As its location suggests it's intended for asserting things about a JSON document in the context of unit tests, so if the specified criteria are met it returns "match", otherwise it throws an exception with a string describing the violated condition. For example:

package require tcltest
package require rltest
package require rl_json
namespace import rl_json::*

tcltest::test foo-1.1 {Assert some things about a JSON doc} -setup {
    set now   [clock seconds]
    set date  [clock format [expr {$now-45}] -format {%Y-%m-%d %H:%M:%S}]
} -body {
    set doc [json template {
        {
            "foo":      "123456",
            "bar":      "some exact value",
            "greet":    "hello, json",
            "sub": [ 
                {"lit":  "?G:foo"},
                {"when": "~S:date"}
            ]   
        }   
    }]  
    rltest compare_json $doc [json template {
        {
            "bar":      "some exact value",         // Require that the string matches exactly
            "foo":      "?R:^[0-9]+$",              // Must match the regex
            "greet":    "?G:hello, *",              // Must match the glob
            "sub": [    
                {"lit":  "?L:?G:foo"},              // Must exactly match "?G:foo", which would otherwise look like a glob match    
                {"when": "?D:within 30 seconds"}    // Must contain a date recognisable to [clock scan] or ISO8601 like 2022-01-30T00:00:00Z that is within 30 seconds of the current time
            ]
        }   
    }]  
} -cleanup {
    unset -nocomplain doc now date
} -result match

tcltest::cleanupTests

Produces:

==== foo-1.1 Assert some things about a JSON doc FAILED
==== Contents of test case:

    set doc [json template {
        {
            "foo":      "123456",
            "bar":      "some exact value",
            "greet":    "hello, json",
            "sub": [
                {"lit":  "?G:foo"},
                {"when": "~S:date"}
            ]
        }
    }]
    rltest compare_json $doc [json template {
        {
            "bar":      "some exact value",         // Require that the string matches exactly
            "foo":      "?R:^[0-9]+$",              // Must match the regex
            "greet":    "?G:hello, *",              // Must match the glob
            "sub": [
                {"lit":  "?L:?G:foo"},              // Must exactly match "?G:foo", which would otherwise look like a glob match
                {"when": "?D:within 30 seconds"}    // Must contain a date recognisable to [clock scan] or ISO8601 like 2022-01-30T00:00:00Z that is within 30 seconds of the current time
            ]
        }
    }]

---- Result was:
left date "2022-08-22 10:03:26" is not within 30 seconds of the current time, at path sub 1 when
---- Result should have been (exact matching):
match
==== foo-1.1 FAILED

rltest.tcl:     Total   1       Passed  0       Skipped 0       Failed  1

Things like optional whitespace and key ordering in objects are not considered a differences between the documents. To only assert certain things about a doc and ignore extra keys, missing keys, or both -subset left, -subset right or -subset intersection are available. The relevant source from rltest is less than 200 lines so it's pretty easy to figure out in the absence of documentation.

It would be easy enough to transplant it over to rl_json, but doing so would require managing some Tcl source in addition to the compiled code which slightly complicates things, and adding parse_args as a dependency.

Also, I notice your rl_json branch adds jsonNRobj to fix the NR issue of the looping commands, that was just a bug in the normal commands which was fixed recently (NR support was added ages ago but some shuffle clearly broke it).

[eric91]

Thanks for pointing me to this code. I'll check if/how I can use it for my needs.