The need for a type::String field and subtypekey for abstract types

[yakir12]

Adding an additional type::String field to all my concrete types – a field that will always contain the name of said concrete type – is irritating, for lack of a better word. I think I understand its purpose, but it would be amazing to avoid the need of adding this "type" field to all our concrete types.

For me this becomes relevant when I need to save a vector that contains elements that can be of any of the sub (concrete) types of a single abstract type. Currently, I just use a Union of all the possible concrete types:

v = JSON3.read(io, Vector{Union{subtypes(MyAbstractType)...}})

and then if I must I can convert that vector of unions to a Vector{MyAbstractType}:

convert(Vector{MyAbstractType}, v)

This seems like a preferable workaround at least for my use-case, but I would love to hear what you think about this "issue" (it is after all more of a gripe).

[yakir12]

hm.. subtypes is not available to me in a module. So I need to manually type out all the possible concrete types. Fine.

[quinnj]

One issue with not requiring the type key in the concrete subtypes is round-tripping; the concrete subtypes themselves aren't marked as being AbstractTypeSubType or whatever, so if they were to get serialized back out to JSON, we couldn't then read them back in. I know it can be a bit onerous, but as you've noted, using Unions is another way to handle this situation.

Alternatively, we could perhaps allow defining concrete subtypes as AbstractTypeSubType, which would be taken into account when serializing, and a type: "ConcreteSubType" would be written for you.

[yakir12]

I have no idea if the following suggestion is silly or not, but here goes: If the goal is to have a record (here in the form of a field) of the name of the type, then wouldn't it be possible that when JSON3.saving something, to retrieve the name of the type of what you're saving, and recording that in some wrapper? Like with getname(x::T) where {T}= string(T)

[quinnj]

Yeah, that's essentially my 2nd proposal; you still have to mark your T as needing to call getname when it saves.

[yakir12]

I'm obviously missing something, but aren't the types retrievable from the call itself? Why do they need to be explicitly declared as strings?

struct Mine
    x::Int
end

getname(x::T) where {T}= string(T)

struct Wrapper
    x
    name::String
    Wrapper(x) = new(x, getname(x))
end

x = Mine(1)

Wrapper(x)

Again, I feel you shouldn't waste your time on this, because I'm convinced I need to read the code and understand what is actually happening.

[jonathan-laurent]

I concur with this thread. A solution with getname that would not require to add a subtypekey field to every subtype seems preferable to me.

[staticfloat]

I wonder if this is solvable by using dispatch on subtypekey(); we could have a default definition of subtypekey(x) = nothing, then during serialization if subtypekey(obj) !== nothing, we can automatically add a mapping of subtypekey(obj) => string(getname(obj)).

[Byrth]

What about something like?

function get_additional_fields(::Type{T}) where T
    if StructType(T) == AbstractType()
            subtype = nothing
            subtypeval = nothing
            for (k,v) in pairs(subtypes(T))
                if isa(obj[name],v)
                    subtypeval = k
                    subtype = v
                    break
                end
            end
            isnothing(subtype) && error('$(typeof(obj[name])) does not inherit from any of the types in StructTypes.subtypes(::Type{$T})')
            [subtypekey(type) => string(subtypeval), get_additional_fields(subtype)...]
    else
        []
    end
end

function fakewrite(obj)
    additional_fields = get_additional_fields(typeof(obj))
    # Do the normal JSON3 stuff and just write the additional fields too
    # remember to check for collisions with existing k=>v pairs, I guess
end

I believe this is analogous to the second approach proposed above.

[robert-wright]

Just thought I'd highlight another use case here. The AbstractType mapping has been very useful for reading JSON data in the CloudEvents format - note the required type field that describes what the event is. My specific use case is building Julia microservices that receive such events.

Many thanks for the efforts put into building JSON3 @quinnj and others - it is excellent.

[quinnj]

I just realized another way, maybe the best way, to resolve this issue using the new-ish CustomStruct functionality. If you have:

abstract type Vehicle end
struct Car <: Vehicle
    make::String
end
StructTypes.StructType(::Type{Vehicle}) = StructTypes.AbstractType()
StructTypes.StructType(::Type{Car}) = StructTypes.CustomStruct()
StructTypes.lower(x::Car) = (type="car", make=x.make)
StructTypes.lowertype(::Type{Car}) = @NamedTuple{type::String, make::String}
Car(x::@NamedTuple{type::String, make::String}) = Car(x.make)
StructTypes.subtypes(::Type{Vehicle}) = (car=Car,)

then it works without needing to have the type field in Car.

[jonathan-laurent]

I like the solution above but I am wondering: are there some standard utilities one can use to avoid writing all the associated boilerplate by hand?

[sairus7]

Got into the same issue. I agree that writing extra code for each subtype and repeating all their field names and types for three times by hand is too much boilerplate. Maybe there is some macro for this?