cadence
cadence copied to clipboard
Measure performance impact of compact codec
Criteria for closing:
- [x] Preliminary performance impact of compact codec vs JSON.
- [ ] Performance impact with real data.
Preliminary performance impact vs the JSON codec is significant:
Bigger speedup and reduction is better for the new codec.
The two tests are with fake values (a string and a resource).
Only the codecs themselves are tested here. The overall performance impact is significantly smaller because encode and decode are only a small part of our overall runtime and memory footprint.
Cadence Value: "This sentence is composed of exactly fifty-nine characters."
Encoding Time (ns):
JSON Average: 4205
JSON Minimum: 791
JSON Maximum: 322958
Value Average: 740
Value Minimum: 208
Value Maximum: 45208
Speedup: 5.6824324324324325
Decoding Time (ns):
JSON Average: 4010
JSON Minimum: 2458
JSON Maximum: 79042
Value Average: 569
Value Minimum: 250
Value Maximum: 23042
Speedup: 7.047451669595782
Size (bytes):
JSON Average: 88
JSON Minimum: 88
JSON Maximum: 88
Value Average: 64
Value Minimum: 64
Value Maximum: 64
Reduction: 1.375
Cadence Value: A.0102030405060708.A.12345678.NFT(Awesome?: true)
Encoding Time (ns):
JSON Average: 2216
JSON Minimum: 750
JSON Maximum: 91708
Value Average: 622
Value Minimum: 375
Value Maximum: 2417
Speedup: 3.562700964630225
Decoding Time (ns):
JSON Average: 4781
JSON Minimum: 2333
JSON Maximum: 60042
Value Average: 682
Value Minimum: 416
Value Maximum: 2916
Speedup: 7.010263929618769
Size (bytes):
JSON Average: 139
JSON Minimum: 139
JSON Maximum: 139
Value Average: 65
Value Minimum: 65
Value Maximum: 65
Reduction: 2.1384615384615384
Cadence Value: A.0102030405060708.A.12345678.NFT(Awesome?: true)
Encoding Time (ns):
JSON Average: 2216 ~ 3x
JSON Minimum: 750 ~ 1x
JSON Maximum: 91708 ~ 120x
How this can be so crazy ranges like this?
Cadence Value: A.0102030405060708.A.12345678.NFT(Awesome?: true) Encoding Time (ns): JSON Average: 2216 ~ 3x JSON Minimum: 750 ~ 1x JSON Maximum: 91708 ~ 120xHow this can be so crazy ranges like this?
I ran it on my macbook. It's likely Goland did something intense during that run.
This is definitely a preliminary test to check if the codec works better at all. Getting it to the point where I could test it took so long that I wanted feedback ASAP.
For the proper test, I intend to use a dedicated machine and real-world transactions. Probably my personal laptop because it's a fairly lean Arch box but maybe we have something in CI that's similar to a production environment?
I believe @fxamacker is gathering real-world transactions to test codec performance much more accurately.
Closing in favor of redesigned solution captured in: https://github.com/onflow/cadence/issues/2157