cairo-vm icon indicating copy to clipboard operation
cairo-vm copied to clipboard

ci: optimize build checks

Open Oppen opened this issue 2 years ago • 4 comments

Speeds up the build checks by sharding the workspace-level feature check and by running each crate-level feature check on its own job.

Checklist

  • [ ] Linked to Github Issue
  • [ ] Unit tests added
  • [ ] Integration tests added.
  • [ ] This change requires new documentation.
    • [ ] Documentation has been added/updated.
    • [ ] CHANGELOG has been updated.

Oppen avatar Aug 17 '23 16:08 Oppen

Benchmark Results for unmodified programs :rocket:

Command Mean [s] Min [s] Max [s] Relative
base big_factorial 2.331 ± 0.051 2.276 2.424 1.02 ± 0.02
head big_factorial 2.296 ± 0.021 2.269 2.333 1.00
Command Mean [s] Min [s] Max [s] Relative
base big_fibonacci 2.358 ± 0.067 2.288 2.484 1.04 ± 0.03
head big_fibonacci 2.257 ± 0.012 2.237 2.270 1.00
Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 8.559 ± 0.112 8.417 8.675 1.00
head blake2s_integration_benchmark 8.716 ± 0.295 8.482 9.315 1.02 ± 0.04
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 2.364 ± 0.027 2.329 2.407 1.00
head compare_arrays_200000 2.385 ± 0.027 2.345 2.433 1.01 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 1.480 ± 0.017 1.457 1.511 1.00 ± 0.02
head dict_integration_benchmark 1.477 ± 0.017 1.453 1.503 1.00
Command Mean [s] Min [s] Max [s] Relative
base field_arithmetic_get_square_benchmark 1.327 ± 0.015 1.314 1.357 1.00
head field_arithmetic_get_square_benchmark 1.334 ± 0.013 1.319 1.363 1.01 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 8.536 ± 0.083 8.448 8.651 1.00
head integration_builtins 8.609 ± 0.102 8.512 8.766 1.01 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 8.914 ± 0.150 8.713 9.252 1.01 ± 0.02
head keccak_integration_benchmark 8.839 ± 0.099 8.729 8.963 1.00
Command Mean [s] Min [s] Max [s] Relative
base linear_search 2.389 ± 0.033 2.341 2.452 1.00
head linear_search 2.405 ± 0.047 2.356 2.502 1.01 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 1.594 ± 0.025 1.566 1.632 1.00
head math_cmp_and_pow_integration_benchmark 1.608 ± 0.037 1.575 1.702 1.01 ± 0.03
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 1.469 ± 0.017 1.439 1.492 1.00
head math_integration_benchmark 1.485 ± 0.051 1.436 1.611 1.01 ± 0.04
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 1.274 ± 0.015 1.260 1.300 1.00
head memory_integration_benchmark 1.286 ± 0.020 1.260 1.310 1.01 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 1.635 ± 0.023 1.615 1.691 1.01 ± 0.02
head operations_with_data_structures_benchmarks 1.626 ± 0.011 1.608 1.636 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 592.3 ± 2.1 589.6 595.1 1.00
head pedersen 601.9 ± 9.7 594.4 626.3 1.02 ± 0.02
Command Mean [ms] Min [ms] Max [ms] Relative
base poseidon_integration_benchmark 983.1 ± 7.4 971.7 998.3 1.00 ± 0.01
head poseidon_integration_benchmark 979.1 ± 8.9 965.6 991.3 1.00
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 1.881 ± 0.009 1.871 1.896 1.00 ± 0.01
head secp_integration_benchmark 1.878 ± 0.018 1.863 1.925 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base set_integration_benchmark 754.6 ± 18.8 742.2 803.5 1.04 ± 0.03
head set_integration_benchmark 725.8 ± 9.5 719.4 752.5 1.00
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 4.705 ± 0.037 4.668 4.784 1.00
head uint256_integration_benchmark 4.736 ± 0.047 4.663 4.794 1.01 ± 0.01

github-actions[bot] avatar Aug 17 '23 17:08 github-actions[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 96.56%. Comparing base (404407d) to head (9cf3dd2).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1384   +/-   ##
=======================================
  Coverage   96.56%   96.56%           
=======================================
  Files          96       96           
  Lines       38488    38488           
=======================================
  Hits        37166    37166           
  Misses       1322     1322           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Aug 17 '23 18:08 codecov[bot]

@juanbono @pefontana we'll need to update the required checks for this to be mergeable.

Oppen avatar Sep 05 '23 14:09 Oppen

Benchmark Results for modified programs :rocket:

Command Mean [s] Min [s] Max [s] Relative
head blake2s_integration_benchmark 11.432 ± 0.041 11.387 11.535 1.00
Command Mean [s] Min [s] Max [s] Relative
head compare_arrays_200000 4.188 ± 0.013 4.172 4.214 1.00
Command Mean [s] Min [s] Max [s] Relative
head dict_integration_benchmark 2.768 ± 0.016 2.743 2.788 1.00
Command Mean [s] Min [s] Max [s] Relative
head field_arithmetic_get_square_benchmark 2.052 ± 0.005 2.047 2.063 1.00
Command Mean [s] Min [s] Max [s] Relative
head integration_builtins 11.203 ± 0.073 11.092 11.346 1.00
Command Mean [s] Min [s] Max [s] Relative
head keccak_integration_benchmark 11.495 ± 0.084 11.346 11.609 1.00
Command Mean [s] Min [s] Max [s] Relative
head linear_search 4.185 ± 0.020 4.158 4.217 1.00
Command Mean [s] Min [s] Max [s] Relative
head math_cmp_and_pow_integration_benchmark 3.050 ± 0.014 3.029 3.075 1.00
Command Mean [s] Min [s] Max [s] Relative
head math_integration_benchmark 2.743 ± 0.015 2.714 2.759 1.00
Command Mean [s] Min [s] Max [s] Relative
head memory_integration_benchmark 2.423 ± 0.011 2.412 2.452 1.00
Command Mean [s] Min [s] Max [s] Relative
head operations_with_data_structures_benchmarks 2.666 ± 0.005 2.659 2.674 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
head pedersen 944.8 ± 3.3 938.4 950.1 1.00
Command Mean [s] Min [s] Max [s] Relative
head poseidon_integration_benchmark 1.771 ± 0.034 1.755 1.867 1.00
Command Mean [s] Min [s] Max [s] Relative
head secp_integration_benchmark 3.443 ± 0.051 3.421 3.586 1.00
Command Mean [s] Min [s] Max [s] Relative
head set_integration_benchmark 1.989 ± 0.007 1.983 2.007 1.00
Command Mean [s] Min [s] Max [s] Relative
head uint256_integration_benchmark 7.485 ± 0.051 7.410 7.595 1.00

github-actions[bot] avatar Sep 05 '23 17:09 github-actions[bot]