csp icon indicating copy to clipboard operation
csp copied to clipboard

[WIP] Setup benchmark suite

Open timkpaine opened this issue 1 year ago • 5 comments

xref: https://github.com/Point72/csp/issues/38 Following @trallard's suggestion, exploring using asv to build a benchmark suite, starting with @AdamGlustein's code in https://github.com/Point72/csp/pull/322#discussion_r1676210203.

Example output, running on just my machine through a few commits:

Screenshot 2024-07-13 at 18 50 25 Screenshot 2024-07-13 at 18 50 34

timkpaine avatar Jul 13 '24 21:07 timkpaine

I had missed the original discussion at #322 , that explains where a lot of this came from. Would be good to tidy up in any case. There are a whole bunch of benchmark tests that I want to write after...

ptomecek avatar Jul 13 '24 23:07 ptomecek

Example from other PR: Screenshot 2024-07-14 at 00 09 53

timkpaine avatar Jul 14 '24 04:07 timkpaine

I had missed the original discussion at #322 , that explains where a lot of this came from. Would be good to tidy up in any case. There are a whole bunch of benchmark tests that I want to write after...

That benchmark script was just something I hacked together, I didn't intend for it to be a canonical example. If we are making standard benchmarks then we should use a more general framework.

AdamGlustein avatar Jul 17 '24 13:07 AdamGlustein

If we are making standard benchmarks then we should use a more general framework.

I did a first pass approximation of a more general framework, extnded to the quantile function to demonstrate https://github.com/Point72/csp/blob/fb36b1b6f448d8c39aaab4edfb4b0ba7eed82ed7/csp/benchmarks/stats/basic.py

timkpaine avatar Jul 17 '24 13:07 timkpaine

@arhamchopra would be good to see if we can use our benchmark infrastructure to run csp benchmarks as well, and maybe fix some of the ugliness I introduced in this PR if we can.

timkpaine avatar Aug 23 '24 16:08 timkpaine