WebSharper.Json: Nicely handle arbitrarily-shaped JSON

[Tarmil]

It would be really good to have a typed way to deserialize arbitrarily-shaped JSON. Here are options to consider:

1. The first possibility that comes to mind would be to handle WebSharper.Core.Json.Value, which is defined like this:

   type Value =
       | Null
       | True
       | False
       | Number of string
       | String of string
       | Array of list<Value>
       | Object of list<string * Value>
   

   This would be easy to implement on the server side (just return the value without any transformation), but the client side is more problematic. We don't want to have to transform the object, but instead we'd like to be able to treat the plain object as the above. This means basically handling this type similarly to the erased Union<...> types. However, several cases are problematic.

   • True and False: they can be merged into a single Bool of bool.
   • Number: On the server side we use a string because it might be deserialized in a variety of different number types, and we need the exact representation to be able to do that faithfully. But on the client we would need it to be float. Not sure how to deal with this.
   • Array: we can replace this with Value[] so that erasure works.
   • Object: here I really don't know what to do to make erasure work nicely.

2. Another possibility would be to simply create a new type with a shape that is more amenable to erasure. That would make things a bit slower on the server side (since we would need to recursively transform the existing Json.Value into this new type, like we do any other type), but fully transparent. Something like this:

   type JsonValue =
       | Null
       | Bool of bool
       | Number of float
       | String of string
       | Array of JsonValue[]
       | Object of JsonObject
   
   and JsonObject =
       member Item : string -> JsonValue with get, set
       member AsObject : JavaScript.Object
       // maybe other members
   

3. Finally, we could just not have a typed representation and instead rely on obj deserialization, as described in #921.

I think option 1's problems aren't really possible to overcome, and that option 2 is the best (which doesn't mean that option 3 shouldn't be implemented too).

[ray440]

I like all those ideas! Looks like JSON is a good format for JavaScript, but inconsistent for other languages.

I like 2) so that inconsistencies can be handled. Maybe as Json.DeserializeJsonValue: string -> JsonValue.

I like 3) so that we are (more) consistent with JSON.Parse. Maybe a wrapper function Json.DeserializeJsonObject: string -> JsonObject , where numbers are floats and everything is an obj.

And I like some of 1). I think it would be more useful to have

type JsonValue = ... 
    | Object of (string*JsonValue)[]

Speculatively, while 3) should be JSON consistent 2) doesn't have to be. We can bypass JSON's shortcomings by defining some interesting types and let people deal with them as they see fit. For example, we could have:

type JsonValue = ...
    // | Number of (string*float?)
    | Number of float
    | Long of long
    | DateTime of DateTime

[Tarmil]

So my idea with 2 was that this could be "free" by implementing it using the same kind of type erasure that we already use for Union<...>. Basically we wouldn't build a value of type JsonValue, instead we compile it specially to directly handle the object as it was passed.

For an example of how this works, currently this:

let myFunc (x: Union<int, string, int[], SomeClass>) =
    match x with
    | Union1Of4 i -> 1
    | Union2Of4 i -> 2
    | Union3Of4 i -> 3
    | Union4Of4 i -> 4

compiles to something like this:

var myFunc = function(x) {
    if (typeof x == 'number') { return 1; }
    else if (typeof x == 'string') { return 2; }
    else if (x instanceof Array) { return 3; }
    else { return 4; }
}

So my proposal was to do something similar:

let myFunc (x: JsonValue) =
    match x with
    | Null -> 1
    | Bool i -> 2
    | Number i -> 3
    | String i -> 4
    | Array i -> 5
    | Object i -> 6

var myFunc(x) {
    if (x == null) { return 1; }
    else if (typeof x == 'boolean') { return 2; }
    else if (typeof x == 'number') { return 3; }
    else if (typeof x == 'string') { return 4; }
    else if (x instanceof Array) { return 5; }
    else { return 6; }
}

Of course this means that we can't just put anything in the union cases, it needs to be a shape that the compiler can erase. Something like Object of (string*JsonValue)[] would require actually building an array, rather than just matching.

All this being said, we can add option 4: have a JsonValue type that we build, rather than erase.