onnxruntime-server icon indicating copy to clipboard operation
onnxruntime-server copied to clipboard

Explore/implement a faster JSON library

Open fullymiddleaged opened this issue 11 months ago • 1 comments

Last one from me, I promise, lol!

I don't have detailed data to back me up, but seeing as the JSON arrays being sent to the server can be quite large (e.g. each request for me is 2 x (1x512) arrays), it might be a good idea to ensure the processing of these is as fast as possible, which when serving multiple requests would be a noticeable upgrade.

Looking at benchmarks, RapidJSON looks like a contender compared to Nlohmann/json. Or maybe Glaze?

https://www.reddit.com/r/cpp/comments/1hdbqrc/whats_the_go_to_json_parser_in_20242025/ https://github.com/miloyip/nativejson-benchmark

fullymiddleaged avatar Jan 21 '25 01:01 fullymiddleaged

Hi @fullymiddleaged :)

Thanks for your good suggestion. I'll review it soon!

kibae avatar Jan 21 '25 18:01 kibae