bayestestR
bayestestR copied to clipboard
Allow Multiple Values of the Log Marginal Likelihood
Describe the solution you'd like
As I mentioned to @bwiernik and @mattansb on twitter yesterday, it appears the bayesfactor_.*
functions only support a single value for the log marginal likelihood. As far as I can tell, the current behavior of the package is to throw a warning if less than 40,000 post-warmup samples are taken but as noted in Schad et al. (2022) and as I discuss in the context of Bayesian Model Averaging here this doesn't necessary guarantee repeated runs of the bridge sampling algorithm will produce stable results and a preferable approach where tractable is to calculate a distribution of estimates for the log marginal likelihood by passing the repetitions
argument to bridgesampling::bridge_sampler
.
I looked into implementing this for brmsfit
objects in #573 since you can store the log marginal likelihood estimates internally in the model object using brms::add_criterion
but it seems like all of the functions that inherent from bayesfactor_models
expect a single estimate for the log ML of each model so for the time being it just takes the median of the stored values. Longer term though, it is probably worth considering adding support for repeated estimates of the log ML.
How could we do it?
As far as implementation is concerned, the first step would be to allow users to pass additional arguments to the call to bridgesampling::bridge_sampler
through bayesfactor_models
. After that things get a bit more complicated because the other bayesfactor.*
functions need to be modified to handle a vector of values for each model. One way to do this that would integrate nicely into the current structure of the print functions might be to simply return the median and quantiles of the bayes factor estimates but I'd need to dig more into the structure of the package to figure out how feasible this is.