Add performance reporting

[ZzEeKkAa]

There is no performance reporting, so we don't know how good we are doing on weekly basis.

Proposal: Create github CI, to run dpbench weekly and add performance report to gh pages using default github runners.

List of metrics to be added:

• Timeseries on implemented tests
• Timeseries on failed tests
• Timeseries on timing for individual test
• Timeseries on timing improvements of numba_dpex against numpy.
• Timeseries on average timing improvements of numba_dpex agains numpy.

Timeseries on average timing improvements of numba_dpex agains numpy.

Could be calculated as avg[ dpex_time/numpy_time - 1 ]. Positive number will indicate improvements.

[diptorupd]

@ZzEeKkAa great initiative, such a feature is essential.

Putting here some of the tools I was referring to when previously I was thinking of this work:

• https://github.com/marketplace/actions/continuous-benchmark
• https://github.com/airspeed-velocity/asv

Few comments on your list of metrics:

• Timeseries on implemented tests

We should decide decide what benchmarks, tests make sense here. I will like to see standard benchmark suites: npbench, Rodhinia, microbenchmarks, and important apps like @fcharras' kmeans implementation

• Timeseries on failed tests

For performance reporting, I do not think it is that useful. Unless, it was a known good test that is now failing or timing out.

• Timeseries on timing for individual test
• Timeseries on timing improvements of numba_dpex against numpy.
• Timeseries on average timing improvements of numba_dpex against numpy.

For the last three we should look at the https://pv.github.io/numpy-bench/ created using airspeedvelocity

Another important question is where do we run the CI run. We can start with github default CPU runners, but I will see if we can do better.