Support for matrix and higher other multidimensional arrays in custom structures

[szafna]

As of current multidimensional Arrays are flattened with JSON.write. In addition JSON.Read by using StructTypes is not able to read (to reshape as defined in structure). I am aware that JSON does not support multidimensional arrays. However thar is some general used practices for writen them in JSON form (at least for 2D it is well known). Therefore, I propose following: -2D arrays can be written in JSON in a ways JSONTables supports -JSON3.read should be able to read/parse 2D Arrays in JSONTables format into custom structure as an alternate way could be that JSON3.read reshape Arrays according to structure definition.

[quinnj]

Note that 2D arrays can already be written using Tables.table and JSONTables.jl, like:

using Tables, JSONTables

tbl = Tables.table(rand(10, 10))
objecttable(tbl)
# or
json = arraytable(tbl)

And then read back in as a "table" and converted back to a matrix like:

original_matrix = jsontable(json) |> Tables.matrix

So perhaps we just need some better documentation in JSONTables and perhaps JSON3 on how to do these operations.

[szafna]

@quinnj , how can be this approach used in case I have Array in a Structure. Ex:

Base.@kwdef mutable struct MyStruct
    par1::Int = 10
    par2::Union{Array{Int,2}, Nothing} = nothing
end

[ueliwechsler]

What you could do is the following:

StructTypes.StructType(::Type{Matrix{N}}) where N<:Number = StructTypes.CustomStruct()
StructTypes.lower(matrix::Matrix{N}) where N<:Number = (content=vec(matrix), size=size(matrix))
StructTypes.lowertype(::Type{Matrix{N}}) where N<:Number = @NamedTuple{content::Vector{N}, size::Tuple{Int, Int}}
function Matrix{N}(matrix::@NamedTuple{content::Vector{N}, size::Tuple{Int, Int}}) where N<:Number 
   return N.(reshape(matrix.content, (matrix.size)))
end

This code results in:

julia> json_str = JSON3.write([1 2; 3 4])
"{\"content\":[1,3,2,4],\"size\":[2,2]}"
julia> JSON3.read(json_str, Matrix{Int})
2×2 Matrix{Int64}:
 1  2
 3  4

But I assume this would be type piracy and therefore not necessarily recommended.

[nlw0]

I just ran into this, in my code I'm relying somewhere with JSON.jl writing matrices as a list of lists, and was surprised with JSON3.jl not following the same design. I understand it's an arbitrary thing, but it would have been nice for me if the behavior was the same.

To add a bit more to the discussion: a proper serialization would probably indeed go the way of storing the shape and the data as a linear vector, especially if you're de-serializing in Julia as well. If this is going to be read by someone else, though, it might be much easier to just access the data as a list-of-lists. Think producing data with Julia in a web server, and using the data in JavaScript.

Since I really want a list-of-lists, I'll probably now have to go with JSON3.write(collect(eachcol(MM))). Could there ever be a more handy way?