numpyro icon indicating copy to clipboard operation
numpyro copied to clipboard

Create a test suite for performance testing

Open neerajprad opened this issue 5 years ago • 0 comments

To get around perf issues, we have had to put in optimizations in certain parts of the codebase. Sometimes these optimizations add to code complexity and can be removed once the perf issues are resolved (either upstream or within numpyro).

It will be nice if we had a set of perf tests capturing different kind of models (ideally something that runs within a couple of minutes):

  • Anytime we have a model that runs slower than expected or we have to put in a hack in the code / interface to get good performance on a model, we can add a corresponding perf test.
  • Then when the issue upstream is resolved or we feel confident that the hack isn't needed, we can refactor and check against the perf test.

I currently use test_mcmc for performance testing, and maybe it is best to stick with that than to add performance tests that we rarely look at. I thought I'll open an issue in case this seems important at a later date.

neerajprad avatar Nov 21 '19 23:11 neerajprad