go-benchmark
go-benchmark copied to clipboard
json-iterator
Re-run benchmarks with recent Go version
Go version 1.11 includes cl47152 and cl98440. The changes when compared to Go version 1.10 are significant.
benchmark old ns/op new ns/op delta
BenchmarkDecodeStdStructMedium-12 32986 26947 -18.31%
BenchmarkEncodeStdStructMedium-12 1569 1192 -24.03%
BenchmarkEncodingJsonStructSmall-12 5451 3699 -32.14%
benchmark old allocs new allocs delta
BenchmarkDecodeStdStructMedium-12 99 8 -91.92%
BenchmarkEncodeStdStructMedium-12 4 2 -50.00%
BenchmarkEncodingJsonStructSmall-12 31 6 -80.65%
I have checked a recent version of go 1.14 (commit ebe49b2c2999a7d4128c44aed9602a69fdc53d16, 2020-02-22) and used benchstat to compare against the current version of jsoniter. I have modified the benchmarks a bit, because the benchmarks in this repo don't fully parse the JSON (incomplete structs). You can find the benchmark I used here.
"old" is encoding/json, "new" is github.com/json-iterator/go (using ConfigCompatibleWithStandardLibrary):
name old time/op new time/op delta
UnmarshalSmall-2 24.9µs ± 1% 4.9µs ± 2% -80.23% (p=0.000 n=10+9)
UnmarshalSmallOnlyId-2 15.4µs ± 1% 4.3µs ± 4% -71.81% (p=0.000 n=10+10)
UnmarshalMedium-2 217µs ± 1% 52µs ± 1% -76.06% (p=0.000 n=10+10)
UnmarshalMediumOnlyIds-2 135µs ± 1% 33µs ± 1% -75.86% (p=0.000 n=8+9)
UnmarshalLarge-2 2.87ms ± 1% 0.80ms ± 2% -72.04% (p=0.000 n=10+10)
UnmarshalLargeOnlyIds-2 1.64ms ± 0% 0.50ms ± 1% -69.27% (p=0.000 n=9+9)
name old alloc/op new alloc/op delta
UnmarshalSmall-2 288B ± 0% 80B ± 0% -72.22% (p=0.000 n=10+10)
UnmarshalSmallOnlyId-2 248B ± 0% 48B ± 0% -80.65% (p=0.000 n=10+10)
UnmarshalMedium-2 1.46kB ± 0% 1.25kB ± 0% -14.29% (p=0.000 n=10+10)
UnmarshalMediumOnlyIds-2 392B ± 0% 400B ± 0% +2.04% (p=0.000 n=10+10)
UnmarshalLarge-2 13.4kB ± 0% 22.5kB ± 0% +66.99% (p=0.000 n=10+8)
UnmarshalLargeOnlyIds-2 402B ± 0% 11937B ± 0% +2869.40% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
UnmarshalSmall-2 8.00 ± 0% 3.00 ± 0% -62.50% (p=0.000 n=10+10)
UnmarshalSmallOnlyId-2 4.00 ± 0% 1.00 ± 0% -75.00% (p=0.000 n=10+10)
UnmarshalMedium-2 49.0 ± 0% 64.0 ± 0% +30.61% (p=0.000 n=10+10)
UnmarshalMediumOnlyIds-2 8.00 ± 0% 51.00 ± 0% +537.50% (p=0.000 n=10+10)
UnmarshalLarge-2 536 ± 0% 1247 ± 0% +132.65% (p=0.000 n=10+10)
UnmarshalLargeOnlyIds-2 35.0 ± 0% 1035.0 ± 0% +2857.14% (p=0.000 n=10+10)
Data initialization (var) in the loop would get us more realistic numbers I think.
Can you even reuse a structure like that? Does Unmarshal clear/zero omitted fields for example?
I'm not sure about the zeroing of omitted fields in consecutive calls of Unmarshal with the same variable. I think, however, that reusing a variable is a realistic use case, since the test-suit of encoding/json includes tests for both scenarios (BenchmarkCodeUnmarshal and BenchmarkCodeUnmarshalReuse).
Here is the same comparison, when re-initializing the target every iteration (you can find the code here):
name old time/op new time/op delta
UnmarshalSmall-2 25.6µs ± 1% 5.9µs ± 1% -77.10% (p=0.000 n=9+9)
UnmarshalSmallOnlyId-2 15.7µs ± 1% 4.5µs ± 2% -71.07% (p=0.000 n=10+10)
UnmarshalMedium-2 245µs ± 1% 64µs ± 2% -73.85% (p=0.000 n=10+10)
UnmarshalMediumOnlyIds-2 135µs ± 3% 33µs ± 1% -75.47% (p=0.000 n=9+10)
UnmarshalLarge-2 3.27ms ± 1% 0.96ms ± 1% -70.57% (p=0.000 n=10+10)
UnmarshalLargeOnlyIds-2 1.71ms ± 1% 0.52ms ± 1% -69.24% (p=0.000 n=8+10)
name old alloc/op new alloc/op delta
UnmarshalSmall-2 400B ± 0% 192B ± 0% -52.00% (p=0.000 n=10+10)
UnmarshalSmallOnlyId-2 264B ± 0% 64B ± 0% -75.76% (p=0.000 n=10+10)
UnmarshalMedium-2 2.50kB ± 0% 2.23kB ± 0% -10.58% (p=0.000 n=10+10)
UnmarshalMediumOnlyIds-2 416B ± 0% 424B ± 0% +1.92% (p=0.000 n=10+10)
UnmarshalLarge-2 30.1kB ± 0% 36.5kB ± 0% +20.97% (p=0.000 n=10+10)
UnmarshalLargeOnlyIds-2 2.09kB ± 0% 12.73kB ± 0% +509.58% (p=0.000 n=10+9)
name old allocs/op new allocs/op delta
UnmarshalSmall-2 9.00 ± 0% 4.00 ± 0% -55.56% (p=0.000 n=10+10)
UnmarshalSmallOnlyId-2 5.00 ± 0% 2.00 ± 0% -60.00% (p=0.000 n=10+10)
UnmarshalMedium-2 67.0 ± 0% 81.0 ± 0% +20.90% (p=0.000 n=10+10)
UnmarshalMediumOnlyIds-2 10.0 ± 0% 53.0 ± 0% +430.00% (p=0.000 n=10+10)
UnmarshalLarge-2 751 ± 0% 1439 ± 0% +91.61% (p=0.000 n=10+10)
UnmarshalLargeOnlyIds-2 82.0 ± 0% 1074.0 ± 0% +1209.76% (p=0.000 n=10+10)
go1.19.13, Considering GC factors, jsoniter is no longer as good as encoding/json.
bench code: https://github.com/win5do/play-go/blob/0c79bdc76dd37be3db9fc69df20c2a922a11ae49/bench/json/bench_test.go
goos: darwin
goarch: arm64
pkg: play-go/bench/json
│ std │ jtr │
│ sec/op │ sec/op vs base │
Unmarshal 1.954µ ± 4% 1.416µ ± 1% -27.54% (p=0.000 n=10)
│ std │ jtr │
│ B/op │ B/op vs base │
Unmarshal 1.376Ki ± 0% 1.360Ki ± 0% -1.14% (p=0.000 n=10)
│ std │ jtr │
│ allocs/op │ allocs/op vs base │
Unmarshal 29.00 ± 0% 34.00 ± 0% +17.24% (p=0.000 n=10)
---
goos: darwin
goarch: arm64
pkg: play-go/bench/json
│ std │ jtr │
│ sec/op │ sec/op vs base │
Unmarshal 17.01µ ± 1% 11.22µ ± 2% -34.03% (p=0.000 n=10)
│ std │ jtr │
│ B/op │ B/op vs base │
Unmarshal 10.10Ki ± 0% 10.99Ki ± 0% +8.86% (p=0.000 n=10)
│ std │ jtr │
│ allocs/op │ allocs/op vs base │
Unmarshal 205.0 ± 0% 268.0 ± 0% +30.73% (p=0.000 n=10)
---
goos: darwin
goarch: arm64
pkg: play-go/bench/json
│ std │ jtr │
│ sec/op │ sec/op vs base │
Unmarshal 241.3µ ± 1% 192.6µ ± 2% -20.19% (p=0.000 n=10)
│ std │ jtr │
│ B/op │ B/op vs base │
Unmarshal 204.5Ki ± 0% 223.4Ki ± 0% +9.28% (p=0.000 n=10)
│ std │ jtr │
│ allocs/op │ allocs/op vs base │
Unmarshal 2.783k ± 0% 3.901k ± 0% +40.17% (p=0.000 n=10)
