JSON3 does not escape dicts of dicts

[rengzhengcodes]

A dictionary that takes dictionaries as keys (very stupid, I know, but it's a quick-and-dirty solution I'm testing), results in improper serialization when writing to file: "(Dict{String, BigFloat}("lat" => , "lon" => ), Dict{String, BigFloat}("lat" => , "lon" => ))"

Where "lat" and "lon" are originally strings.

[quinnj]

Yeah, we've never really supported non-string-like things for object keys because we short-cut by just calling string(x) on whatever the key is supposed to be. I'll have to think through this a bit more, but we might be able to do something better here.

[rengzhengcodes]

Thanks! Json.write() seems to exhibit closer to the intended behavior, but the read seems to have issues. Given Stringify is extensionally equivalent for base-objects, it would be rather unsafe but plausible to put a parse clause around the keys instantiating them I believe. Probably could work better with type detection then parsing the arguments but that's a digression.

I'll try to figure something out and come back with a solution for those that make their way here from when this ends up being indexed by google.

[guimarqu]

Yeah, we've never really supported non-string-like things for object keys because we short-cut by just calling string(x) on whatever the key is supposed to be. I'll have to think through this a bit more, but we might be able to do something better here.

JSON only allows strings as keys, so I can understand that JSON3's default behavior is to shortcut by just calling string(x). JSON3 should at least throw an error message if it cannot stringify the key (see e.g. https://github.com/quinnj/JSON3.jl/issues/285).

Here is my workaround when I work my data structures involving dictionaries using non-string keys (it won't work with a dictionary as a key) :

# JSON3 does not support Dict when the type of a key is not String.
# Consider the following type.

struct MyType
    dict1::Dict{Tuple{Int,Int},String}
    dict2::Dict{Tuple{Int,Int,Int}, String}
end

# Serialization / Deserialization of MyType goes through a custom type.
StructTypes.StructType(::Type{MyType}) = StructTypes.CustomStruct()

# You need to define the intermediate types that will be the JSON schema of your type.
struct EntryWrapper{K,V}
    key::K
    value::V
end

StructTypes.StructType(::Type{EntryWrapper{K,V}}) where {K,V} = StructTypes.Struct()

struct MyTypeJSONRepr
    dict1::Vector{EntryWrapper{Tuple{Int,Int}, String}}
    dict2::Vector{EntryWrapper{Tuple{Int,Int,Int}, String}}
end

StructTypes.StructType(::Type{MyTypeJSONRepr}) = StructTypes.Struct()

# Define the following constructors to instantiate the intermediate structure from the initial 
# one and the other way around.
wrap_entries(dict::Dict{K,V}) where {K,V} = [EntryWrapper(k, v) for (k, v) in dict]
unwrap_entries(vec::Vector{EntryWrapper{K,V}}) where {K,V} = Dict{K,V}(elem.key => elem.value for elem in vec)

MyTypeJSONRepr(initial::MyType) = MyTypeJSONRepr(
    wrap_entries(initial.dict1),
    wrap_entries(initial.dict2)
)

MyType(repr::MyTypeJSONRepr) = MyType(
    unwrap_entries(repr.dict1),
    unwrap_entries(repr.dict2)
)


# MyType must be written as a MyTypeJSONRepr.
StructTypes.lower(initial::MyType)  = MyTypeJSONRepr(initial)

# MyType must be read from a MyTypeJSONRepr object.
StructTypes.lowertype(::Type{MyType})  = MyTypeJSONRepr




example = MyType(
    Dict((1,2) => "3", (4, 5) => "6"),
    Dict((10, 11, 12) => "13", (14, 15, 15) => "17")
)
json = JSON3.write(example)
read_example = JSON3.read(json, MyType)