Performance

[gjroelofs]

After Jackson's comparison (Speed and GC) on the performance of different serialization frameworks it might be interesting to take a critical look at the overall performance of FullSerializer.

Any thoughts on where we could best start with improving this?

[jacobdufault]

I expect performance to be significantly better if Full Serializer is used in AOT mode - that should be benchmarked though.

One of the reasons FS has poor performance is due to cyclic graph support. This requires that we support writing into previous serialization state, which means that serialization cannot write directly to a memory buffer (while, it could, but then all of the then-serialized state would need to get pushed forward). I have some ideas for optimizations here, ie, write serialized state into chunks so that the push-forward cost is greatly reduced.

All in all though, FS is all about being easy to use. Unity, LitJSON, and Json.NET all fail on various different serialization scenarios, whereas FS just keeps humming. Interestingly, performance here is related directly to the flexibility of each serializer - though I'm surprised Unity's serialization time isn't all that much faster than LitJSON.

[gjroelofs]

Completely true, but there are still quite some cases that could be optimized.

A simple example would be aliasing on types which could shave off quite some time on parsing and file size. (Correct me if I'm mistaken, as I've not found this implemented yet)

[jacobdufault]

A simple example would be aliasing on types which could shave off quite some time on parsing and file size. (Correct me if I'm mistaken, as I've not found this implemented yet)

Can you explain this optimization? Type lookup should be cached already using fsTypeCache. Or do you mean something like

{
    "$type_map": {
        "Hello.MyType": "1",
        "Hello.SecondType": "2"
    },
    "object": {
        "$type": "1"
    }
}

One problem with this optimization is that it makes the JSON files harder to hand-edit.

[gjroelofs]

Yes, I meant aliasing on types in the JSON. I would actually argue that hand editing could be simpler if you know the mapping. Typing a small alias over a full namespaced type is arguably easier. (I agree that discovery as you move through the JSON to find the location you would need to edit would be harder; but normally people would use search for any reasonably sized JSON at that point)

[jacobdufault]

f43c7f9b0f850e23676c09e173dd98cc8fef67ff contains a number of AOT quality of life improvements. Inside of Unity, AOT candidate types are automatically discovered and only a single button press is needed to generate the models. AOT models are versioned and if they are stale the standard reflection-based serialization pipeline is used automatically.

I think the end goal for AOT models is that they bypass almost the entire serialization pipeline and directly emit fsData instances (or even beyond that, directly emit JSON and removing the fsData abstraction, as it is actually introduces significant performance regressions).