elasticlient icon indicating copy to clipboard operation
elasticlient copied to clipboard

Json parsing speed too slow

Open johniez opened this issue 5 years ago • 0 comments

JsonCpp library used to parse scroll results is extremely slow. Scrolling through larger indices show the delay much more. In numbers: what the current implementation iterates in 85 seconds, scroll implemented using rapidjson did in 25 seconds.

Unfortunately JsonCpp is used in Scroll header interface, so the change would break API/ABI. But the speed is worth it.

Possible implementation with rapidjson (as it is header only library):

  • hide it completely inside elasticlient, without publishing it in public interface
  • make the rapidjson parsing public (header only), so each client could compile it again

First option means that parsing has to be done twice. Internally and then for data parsing itself. Otherwise there is a risk of using different versions (binary incompatible) of rapidjson. The second option allows shared parsing (internall and client's data) without possibility of using different versions of rapidjson (one compiled into elasticlient, another version in client code).

johniez avatar Jul 11 '19 09:07 johniez