dowhy
dowhy copied to clipboard
Increase the default number of simulations
I would like to suggest changing the default values for the number of simulations:
Currently, the number of simulations for the Confidence Intervals is set to 100:
# The default number of simulations for statistical testing
DEFAULT_NUMBER_OF_SIMULATIONS_STAT_TEST = 1000
# The default number of simulations to obtain confidence intervals
DEFAULT_NUMBER_OF_SIMULATIONS_CI = 100
# The portion of the total size that should be taken each time to find the confidence intervals
# 1 is the recommended value
# https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/readings/MIT18_05S14_Reading24.pdf
# https://projecteuclid.org/download/pdf_1/euclid.ss/1032280214
This is too little. I suggest to use at least 399 instead. IMHO, 999 would be even better and preferable as default.
This paper can be used as reference. https://www.econstor.eu/bitstream/10419/67820/1/587473266.pdf
I already open another issue: https://github.com/py-why/dowhy/issues/809 relates to the same part of the code. I would be probably most convenient to change both.
Good point, and thanks for providing the reference. It was set to 100 primarily to avoid computational burden for a new user. Let's set it to 399. Feel free to open a PR (do also add the paper link as an inline comment to help future readers of the code).