query-json icon indicating copy to clipboard operation
query-json copied to clipboard

Slower than jq?

Open unhammer opened this issue 3 years ago • 1 comments

jq Version: 1.6-1ubuntu0.20.04.1 and query-json from the github sep 2020 release

$ wget -q https://github.com/json-iterator/test-data/raw/master/large-file.json 

$ time jq . large-file.json >/dev/null

real	0m1,437s
user	0m1,260s
sys	0m0,040s

$ time query-json . large-file.json >/dev/null

real	0m2,164s
user	0m1,868s
sys	0m0,153s

$ jq --version
jq-1.6

$ query-json --version
0.5.20

Maybe jq got faster since q was written?

Also, is there a way to stream like https://teddit.net/r/bash/comments/myoft4/streaming_with_jq/ ? So one can process files like https://dumps.wikimedia.org/wikidatawiki/entities/latest-lexemes.json.bz2

unhammer avatar May 03 '22 11:05 unhammer

Hey @unhammer.

jq and query-json on big files are very close on performance, from the start https://github.com/davesnx/query-json/blob/main/benchmarks/report.md#simple-operation-keys-an-attribute-on-a-huge-json-110m-file

Streaming isn't implemented on query-json, but would be a great addition.

davesnx avatar May 03 '22 13:05 davesnx

Closing this since isn't "an issue"

davesnx avatar Jan 30 '23 08:01 davesnx