stacks-core icon indicating copy to clipboard operation
stacks-core copied to clipboard

Chore: Replace Hex Encoding/Decoding

Open ASuciuX opened this issue 1 year ago • 5 comments

Description

I took a look at the functions that were doing the hex encoding and decoding in the stacks-common package and there was an easy replacement with an external crate (faster-hex) in order to improve the performance. The results show a performance increase of about 1% when processing blocks:

$ hyperfine -w 3 -r 10 "./stacks-inspect-hex-modifications replay-block ~/chainstate/ range 99990 100000" "./stacks-inspect-9385b4236 replay-block ~/chainstate/ range 99990 100000"
Benchmark 1: ./stacks-inspect-hex-modifications replay-block ~/chainstate/ range 99990 100000
  Time (mean ± σ):      7.956 s ±  0.036 s    [User: 7.571 s, System: 0.352 s]
  Range (min … max):    7.929 s …  8.052 s    10 runs
 
Benchmark 2: ./stacks-inspect-9385b4236 replay-block ~/chainstate/ range 99990 100000
  Time (mean ± σ):      8.023 s ±  0.021 s    [User: 7.610 s, System: 0.379 s]
  Range (min … max):    7.992 s …  8.061 s    10 runs
 
Summary
  ./stacks-inspect-hex-modifications replay-block ~/chainstate/ range 99990 100000 ran
    1.01 ± 0.01 times faster than ./stacks-inspect-9385b4236 replay-block ~/chainstate/ range 99990 100000

ASuciuX avatar Mar 01 '24 13:03 ASuciuX

@jcnelson @kantai From the benchmarking results, the implementation with faster-hex is better, but also it adds an extra dependency. Please let me know if you think this is worth adding.

ASuciuX avatar Mar 01 '24 13:03 ASuciuX

Might also consider the hex-simd package which is ~2-3x faster than faster-hex https://nugine.github.io/simd-benches/

zone117x avatar Mar 04 '24 11:03 zone117x

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 78.19%. Comparing base (383d586) to head (3317695). Report is 19 commits behind head on next.

Additional details and impacted files
@@            Coverage Diff             @@
##             next    #4465      +/-   ##
==========================================
- Coverage   83.46%   78.19%   -5.28%     
==========================================
  Files         448      448              
  Lines      324321   324311      -10     
==========================================
- Hits       270698   253585   -17113     
- Misses      53623    70726   +17103     
Files Coverage Δ
stacks-common/src/util/hash.rs 80.60% <100.00%> (-0.57%) :arrow_down:
stacks-common/src/util/macros.rs 83.98% <100.00%> (+0.12%) :arrow_up:
stacks-common/src/util/mod.rs 44.82% <ø> (-8.63%) :arrow_down:
stackslib/src/blockstack_cli.rs 77.37% <100.00%> (-2.79%) :arrow_down:

... and 165 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 383d586...3317695. Read the comment docs.

codecov[bot] avatar Mar 04 '24 12:03 codecov[bot]

Per the call today, I think we should defer further consideration of this until we remove hex encode/decode from the DB I/O paths (since that's where a lot of this happens, and in this case, we can just remove the hex codec altogether).

jcnelson avatar Mar 04 '24 16:03 jcnelson

Per the call today, I think we should defer further consideration of this until we remove hex encode/decode from the DB I/O paths (since that's where a lot of this happens, and in this case, we can just remove the hex codec altogether).

I'll convert this into draft till then.

ASuciuX avatar Mar 05 '24 14:03 ASuciuX