Document serializers and their usages

[StachuDotNet]

trafficstars

We use a number of serializers through our backend. Some are System.Text.Json, some are Newtonsoft, and some are some wacky custom thing[1].

[1] I think these still exist - I'll have to find them still.

We have a number of issues with the current arrangement:

• the custom wacky serializers should go
• updating any serializer feels risky - we don't have a centralized way of
• we'd like to generally migrate everything to System.Text.Json; it's the future of .NET, performs much better, especially in Analysis (in WASM)

[StachuDotNet]

• [x] gather some serializers by searching in code for "newtonsoft"
• [x] gather some serializers by searching in code for "system.text.json"
• [ ] read through the changelog for some notes on usages
• [ ] try to find the custom serializers
• [ ] document the usages

[StachuDotNet]

@pbiggar from this comment:

Perhaps we could mark each deserialization with some token (such as DARK_SERIALIZATION) and make some categories based on how safe it is to change.

I don't know what this means. Where/how would we "mark each deserialization with some token"? Is that referring to something in code, or something within the output of this Issue (i.e. some document)?

[pbiggar]

I meant some way of marking serializers in code so we can find them again. Similar to TODO, CLEANUP, except that the purpose is not necessarily to remove them.

Potentially they could also have metadata (whether it's buggy for example, or slow, or we'd like to stop using it, or that this one is the future for some set of use cases). Or maybe just comments.

[pbiggar]

You'll find some custom serializers in libjwt, and many in dvalrepr*.fs

[StachuDotNet]

Serializers/deserializers I've found, all under fsharp-bacikend/src:

• ApiServer/Api/Json.fs
• BackendOnlyStdLib/LibJwt.fs
  • has ofJson and parseJson - these are Newtonsoft-based
  • has a Legacy module with type Yojson, toYojoson
• LibBackend/QueueSchedulingRules.fs
  • has STJJsonConverter
  • has JsonConverter (which is Newtonsoft)
• LibExecution/DvalReprExternal has a bunch
• Prelude/Prelude.fs has Vanilla and OCamlCompatible
• LibExecution/DvalReprInternalDeprecated has writeJson and writePrettyJson. lots of custom stuff in here
• LibExecutionStdLib/LibObject.fs has PrettyResponseJsonV0

I'm going to just edit this comment for a while, including some notes of usages, while I process.

[StachuDotNet]

Regarding the markups, maybe we have two variants:

e.g.

• SERIALIZER_DEF [Newtonsoft/STJ/Custom] [name]
• SERIALIZER_USAGE [Newtonsoft/STJ/Custom] [name]

We can then focus on reducing the number of SERIALIZER_USAGE Custom results, etc. items with STJ can hang around indefinitely.

We could trickle the USAGEs up the the dependency graph

[pbiggar]

Good format.

Yeah, any SERIALIZER_USAGE Custom or SERIALIZER_USAGE Newtonsoft should be replaced by STJ serializers if possible, and should be deprecated if they're part of the stdlib.

[pbiggar]

How are we doing on this?

[StachuDotNet]

We've removed the OCaml-compatible serializer from Prelude, which is great.

I can't say I'm feeling particular pains with the remaining serializers lately. We should create new/better LibJWT fns (which we've ticketed here), but besides that, we seem to be in a good place? I'd be fine closing this Issue, maybe after removing the annotations we added?

[pbiggar]

Maybe what we should do is create a serializers doc in the repo that just names all the serializers, based on the info in this post. I'd be happy to close it afterwards.

[pbiggar]

I think we can call this closed