numcodecs Continuous benchmarking with ASV

It would be great to implement ASV or similar so we catch any changes that introduce a performance regression.

Nov 19 '18 10:11 alimanfoo

@joshmoore this was something we were looking at right?

Oct 29 '22 20:10 jakirkham

@jakirkham, yes, during the last GSOC round. There are a few decisions to be made and then various functions to be written, but pretty straight-forward.

Oct 30 '22 20:10 joshmoore

Just FYI, it's worth having a look at upstream discussions about the future of the project (e.g. https://github.com/airspeed-velocity/asv/issues/1219 ) before committing to ASV

Oct 31 '22 09:10 jeromekelleher

Oh wow. Thanks for the heads up, @jeromekelleher! (Yet another reason it was sad to have missed the summit.) I'll keep an eye on it, but in our testing with asv so far (for zarr-python) there was already one "speed regression" identified. (By another "j" programmer who shall rename nameless.) So it might make sense to get something in place soonish, with the understanding that we will be up for jumping ship when the time comes.

Oct 31 '22 18:10 joshmoore

Just following up on this: Were any benchmarks implemented for numcodecs? If so, did the benchmarks use ASV?

it's worth having a look at upstream discussions about the future of the project (e.g. airspeed-velocity/asv#1219 ) before committing to ASV

For context: I'm currently researching benchmarking tools for benchmarking zarr-python. ASV looked good. Until I saw that discussion!

Sep 18 '23 15:09 JackKelly

In the context of GSOC asv was only implemented for zarr-python and there only superficially.

Sep 22 '23 22:09 joshmoore