Use a different JSON library

[benjaminjkraft]

There are a few potential ideas here, which may coordinate or conflict.

First, any code that's using genqlient should be a great match for one of the libraries that does JSON via codegen rather than reflection, which are theoretically way better for perf. @StevenACoffman and @csilvers pointed me to easyjson, ffjson, go-codec, ~jsonparser,~ and gojay, and Steve posted some more details about them below; I'd also want to run our own benchmarks on actual genqlient-generated code.

On the other hand, right now we do a lot of work to try to do what we want within the encoding/json API, so it might instead be better to use a library that has a different set of extension points more suitable to our needs. Steve pointed me to mapstructure which is definitely aimed at what we are trying to do in the case of abstract types and custom unmarshalers, which are the most complex case on the decoding side. Sadly it looks like it has somewhat limited support for encoding (see mitchellh/mapstructure#166), and while it has more flexible support for embedding than the standard library, it will only simplify our unmarshalers (by making it easier to merge embedded structs), not eliminate them.

Switching to mapstructure would also preclude switching to one of the other code-gen libraries, which tend to be compatible with encoding/json. Some do add additional options, so we could explore that. (Of those linked above, go-codec seems to have the most extension points, and would likely have similar benefits to mapstructure, although like mapstructure its support for embedding is likely not flexible enough to solve all our problems. It also has support for other encodings, which could be useful for e.g. putting your responses in a cache, or if you use some strange transport for your GraphQL (not likely). Note that it's possible golang/go#5901 will add some of the features we'd like to the standard library, although the timeline on that is very unclear to me.)

One potential future need comes from #30; which for fields will require special handling in (un)marshaling to (un)marshal several layers of JSON structure into/from one layer of Go structure. I didn't see support from any of the libraries for that, although mapstructure might at least provide better tools for us to do it ourselves, or one could imagine any of the libraries might add it (essentially what we want is support for a tag json:"a.b.c"). Another potential future need is more options for how to handle omitempty; right now we match JSON because I think that's the least-bad default (see #103) but it's got some problems for sure. (And now we probably do want to try to keep that behavior, at least as the default, for better or worse, which could be tricky for libraries that have different behavior.)

Finally, as mentioned in #120, another option is to roll our own, making use of what we know about our types to simplify things. I suspect it's not worth it yet, but if our (un)marshalers start having to get even more complicated, it may be; we don't need to implement nearly all of the behavior of the general-purpose libraries since we know what we want our types to look like.

For any of the third-party codegen options, we do need to make sure this won't cause trouble with custom scalars: it might be a little surprising if you have to rerun genqlient because you changed a json tag in a custom scalar, although it's not a huge problem in practice I expect (especially since probably most custom scalars will have UnmarshalJSON methods). In any case we could also put this behind a flag. A first-party implementation could call out to encoding/json for custom scalars, of course.

[benjaminjkraft]

As mentioned in #120, another option is to roll our own, making use of what we know about our types. I suspect it's not worth it yet, but if our (un)marshalers start having to get even more complicated, it may be, since we end up doing a lot of work to get encoding/json to do what we want.

[csilvers]

go-codec also supports codegen for json (and other encoding formats as well) I believe, via codecgen.

[StevenACoffman]

ugorji/go-codec is worth adding to the list above. It is described here. Supported Serialization formats are:

• msgpack: https://github.com/msgpack/msgpack
• binc: http://github.com/ugorji/binc
• cbor: http://cbor.io http://tools.ietf.org/html/rfc7049
• json: http://json.org http://tools.ietf.org/html/rfc7159

I hadn't mentioned it before, because I'm not sure that anything but json is relevant to us, and so it's possible part of ugorji/go popularity is only due to non-relevant formats. However, if it's well maintained, and has the right features, it's worth considering.

library	order by page rank	number of github stars
go-codec	145	1371
github.com/mailru/easyjson	185	3331
github.com/buger/jsonparser	338	4185
github.com/pquerna/ffjson	1281	2787
github.com/francoispqt/gojay	1335	1999

I like to use popularity as a proxy for package quality, but comparing other things like contribution rate is a little more work.

[benjaminjkraft]

I've updated the issue description to add the wider range of considerations we've added here, and some notes from a quick look at a few of the options.