starknet-devnet-rs icon indicating copy to clipboard operation
starknet-devnet-rs copied to clipboard

Try out cairo-native

Open FabijanC opened this issue 4 months ago • 1 comments

As described here: https://spaceshard.slack.com/archives/C029F9AN8LX/p1754994098998129

FabijanC avatar Aug 12 '25 14:08 FabijanC

The following was originally posted on Slack. Waiting for comments.

I ran a benchmark, which sends 500 minting requests to Devnet (internally, this produces an invoke tx to STRK ERC20). I ran this 10 times for the latest Devnet commit on main (uses blockifier 0.15.0-rc.2) and 10 times for Devnet with cairo_native enabled (uses blockifier 0.15.0-rc.3). Both Devnets were run as Docker containers. Here's what I got (the measured values are in seconds):

  • Latest Devnet:
Time sample:   DescribeResult(nobs=10, minmax=(64.17280006408691, 66.57202363014221), mean=65.35412051677704, variance=0.6618370423397418, skewness=-0.3165210792540387, kurtosis=-1.123343669071298)
  • Devnet with cairo_native:
Time sample:   DescribeResult(nobs=10, minmax=(61.838404178619385, 64.88976907730103), mean=63.579264760017395, variance=1.0566302223600166, skewness=-0.3388753169287398, kurtosis=-1.099763810594258)

I ran an independent t-test, with alternative hypothesis set to "greater" (testing the hypothesis that latest Devnet uses more time):

TtestResult(statistic=4.281465331376874, pvalue=0.00022459226077571598, df=18.0)

FabijanC avatar Aug 27 '25 07:08 FabijanC