EpiNow2 icon indicating copy to clipboard operation
EpiNow2 copied to clipboard

Change Getting started documentation to point users to faster models

Open jamesmbaazam opened this issue 6 months ago • 5 comments

This issue can be solved after merging the benchmarking vignette in #695.

I was curious about how the paper "Real-time estimation of the epidemic reproduction number: Scoping review of the applications and challenges" measured run times of the various R packages and came across this section in the supplementary material where more details are given (page 3):

Computational speed has been assessed by each author of this study with different computer specifications (see Table B). The main function in each package/tool was taken from provided examples (if available) and wrapped in the system.time() R function to measure the execution time. The main function estimated the reproduction number for each R package/tool except for epicontacts, which estimates the serial interval. We chose the following classifications: <10 seconds = very good, 10 seconds – 5 minutes = good, >5 minutes = poor. The classification allocated to each package was based on the agreement of at least 2 out of the 3 computers. We note that such direct comparisons of the runtimes of the different models may not be fair, as the examples provided by each package which we have used to assess speed vary in terms of the dataset used, model complexity, and dimensionality of the reproduction number to estimate. Nevertheless, we assume that examples will always be relatively simple and therefore their computational speed may be a good overall indicator of speed of reproduction number estimation in general using a given package.

This has got me thinking about whether we should change our docs to use the quicker models that sacrifice accuracy as that is what users will interact with first (copy & paste to try out) but with a caveat. We can then signpost to the slower but more accurate models for real-world use cases.

I'm also making a note here to raise an issue in EpiEstim to re-assess the speed score in this table using other faster and relatively accurate models with evidence in #695.

jamesmbaazam avatar Jul 31 '24 13:07 jamesmbaazam