onnxruntime-server
onnxruntime-server copied to clipboard
Explore/implement a faster JSON library
Last one from me, I promise, lol!
I don't have detailed data to back me up, but seeing as the JSON arrays being sent to the server can be quite large (e.g. each request for me is 2 x (1x512) arrays), it might be a good idea to ensure the processing of these is as fast as possible, which when serving multiple requests would be a noticeable upgrade.
Looking at benchmarks, RapidJSON looks like a contender compared to Nlohmann/json. Or maybe Glaze?
https://www.reddit.com/r/cpp/comments/1hdbqrc/whats_the_go_to_json_parser_in_20242025/ https://github.com/miloyip/nativejson-benchmark
Hi @fullymiddleaged :)
Thanks for your good suggestion. I'll review it soon!