cairo_native icon indicating copy to clipboard operation
cairo_native copied to clipboard

[DNM] Execution trace dumping with contract support.

Open azteca1998 opened this issue 1 year ago • 3 comments

Checklist

  • [ ] Linked to Github Issue
  • [ ] Unit tests added
  • [ ] Integration tests added.
  • [ ] This change requires new documentation.
    • [ ] Documentation has been added/updated.

azteca1998 avatar Aug 28 '24 20:08 azteca1998

Benchmarking results

Benchmark for program dict_insert

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 21.195 ± 0.154 21.036 21.575 100.81 ± 1.39
cairo-native (embedded AOT) 3.985 ± 0.017 3.949 4.010 18.95 ± 0.24
cairo-native (embedded JIT using LLVM's ORC Engine) 3.827 ± 0.023 3.788 3.871 18.20 ± 0.24
cairo-native (standalone AOT with -march=native) 0.210 ± 0.002 0.206 0.215 1.00

Benchmark for program dict_snapshot

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 5.739 ± 0.036 5.693 5.808 18.21 ± 0.13
cairo-native (embedded AOT) 4.156 ± 0.038 4.113 4.232 13.19 ± 0.13
cairo-native (embedded JIT using LLVM's ORC Engine) 4.005 ± 0.020 3.978 4.037 12.71 ± 0.08
cairo-native (standalone AOT with -march=native) 0.315 ± 0.001 0.314 0.316 1.00

Benchmark for program factorial_2M

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 14.283 ± 0.033 14.234 14.329 10.58 ± 0.03
cairo-native (embedded AOT) 4.215 ± 0.025 4.183 4.253 3.12 ± 0.02
cairo-native (embedded JIT using LLVM's ORC Engine) 4.031 ± 0.027 3.996 4.079 2.99 ± 0.02
cairo-native (standalone AOT with -march=native) 1.350 ± 0.002 1.348 1.353 1.00

Benchmark for program fib_2M

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 14.679 ± 0.105 14.578 14.892 188.63 ± 1.85
cairo-native (embedded AOT) 3.762 ± 0.021 3.734 3.790 48.34 ± 0.42
cairo-native (embedded JIT using LLVM's ORC Engine) 3.555 ± 0.019 3.533 3.585 45.69 ± 0.39
cairo-native (standalone AOT with -march=native) 0.078 ± 0.001 0.077 0.081 1.00

Benchmark for program linear_search

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 5.744 ± 0.035 5.688 5.799 2902.76 ± 66.70
cairo-native (embedded AOT) 3.860 ± 0.013 3.846 3.888 1950.84 ± 43.69
cairo-native (embedded JIT using LLVM's ORC Engine) 3.752 ± 0.014 3.736 3.778 1896.06 ± 42.55
cairo-native (standalone AOT with -march=native) 0.002 ± 0.000 0.002 0.002 1.00

Benchmark for program logistic_map

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
Cairo-vm (Rust, Cairo 1) 5.663 ± 0.035 5.630 5.740 23.31 ± 0.27
cairo-native (embedded AOT) 3.957 ± 0.021 3.933 3.994 16.29 ± 0.18
cairo-native (embedded JIT using LLVM's ORC Engine) 3.857 ± 0.021 3.827 3.904 15.88 ± 0.18
cairo-native (standalone AOT with -march=native) 0.243 ± 0.002 0.242 0.250 1.00

github-actions[bot] avatar Aug 28 '24 21:08 github-actions[bot]

❌ Code is not formatted! Please run cargo format and push the changes.

github-actions[bot] avatar Sep 10 '24 15:09 github-actions[bot]

Benchmark results Main vs HEAD.

Command Mean [s] Min [s] Max [s] Relative
base dict_insert.cairo (JIT) 3.779 ± 0.023 3.748 3.819 1.00
base dict_insert.cairo (AOT) 3.939 ± 0.036 3.894 3.994 1.04 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head dict_insert.cairo (JIT) 3.933 ± 0.052 3.870 4.015 1.00
head dict_insert.cairo (AOT) 4.099 ± 0.038 4.026 4.147 1.04 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base dict_snapshot.cairo (JIT) 3.968 ± 0.023 3.934 4.008 1.00
base dict_snapshot.cairo (AOT) 4.139 ± 0.036 4.093 4.205 1.04 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head dict_snapshot.cairo (JIT) 4.117 ± 0.036 4.064 4.189 1.00
head dict_snapshot.cairo (AOT) 4.239 ± 0.029 4.207 4.280 1.03 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base factorial_2M.cairo (JIT) 3.997 ± 0.027 3.953 4.050 1.00
base factorial_2M.cairo (AOT) 4.191 ± 0.038 4.150 4.288 1.05 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head factorial_2M.cairo (JIT) 4.125 ± 0.031 4.079 4.171 1.00
head factorial_2M.cairo (AOT) 4.331 ± 0.043 4.268 4.419 1.05 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base fib_2M.cairo (JIT) 3.553 ± 0.023 3.520 3.590 1.00
base fib_2M.cairo (AOT) 3.726 ± 0.021 3.680 3.756 1.05 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head fib_2M.cairo (JIT) 3.681 ± 0.033 3.628 3.729 1.00
head fib_2M.cairo (AOT) 3.915 ± 0.043 3.858 3.961 1.06 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base linear_search.cairo (JIT) 3.739 ± 0.027 3.683 3.780 1.00
base linear_search.cairo (AOT) 3.828 ± 0.019 3.808 3.854 1.02 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head linear_search.cairo (JIT) 3.891 ± 0.032 3.848 3.940 1.00
head linear_search.cairo (AOT) 4.000 ± 0.040 3.935 4.049 1.03 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base logistic_map.cairo (JIT) 3.847 ± 0.016 3.828 3.880 1.00
base logistic_map.cairo (AOT) 3.948 ± 0.040 3.886 4.038 1.03 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
:--- ---: ---: ---: ---:
head logistic_map.cairo (JIT) 3.981 ± 0.031 3.914 4.034 1.00
head logistic_map.cairo (AOT) 4.051 ± 0.029 4.002 4.091 1.02 ± 0.01

github-actions[bot] avatar Oct 08 '24 16:10 github-actions[bot]

Codecov Report

Attention: Patch coverage is 82.45614% with 10 lines in your changes missing coverage. Please review.

Project coverage is 80.72%. Comparing base (e41f7b1) to head (8411019). Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/compiler.rs 82.45% 10 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #775      +/-   ##
==========================================
- Coverage   80.76%   80.72%   -0.04%     
==========================================
  Files         107      107              
  Lines       29256    29290      +34     
==========================================
+ Hits        23628    23644      +16     
- Misses       5628     5646      +18     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Nov 19 '24 17:11 codecov-commenter