Question: Asv to track algorithm performance and not just timing

[AKuederle]

This is not an issue, but rather a question, if asv is the right tool for what I want to do.

I am currently developing a library that implements a set of algorithms to perform time series analysis. For this project I am looking for a way to present the performance of these algorithms (prediction, forecasting performance, ...) on some benchmark dataset for vairous parameter combinations. Ideally, I would like to be able to present these results on a static website (just like asv does it) and be able to rerun the benchmarks once there are changes to the algorithms.

I saw in the documentation that it is possible to track arbitrary parameters with asv. However, as asv is focused on benchmarking time and memory consumption, I am not sure if this is will end up to be way to hacky.

So simply put, does anyone has experience with using asv for this usecase or would you recommend using something else/build something custom for my project.

Thanks in advance!

[fcooper8472]

I have a couple of very similar use-cases in mind and would also be very interested in the answer to this question.