Feature Suggestion: Bayesian Bootstrap

[ParadaCarleton]

The Bayesian bootstrap simulates an approximation for the posterior distribution of a parameter by assuming that only the observed values are possible, and the relative frequency of each observed value follows a Dirichlet distribution with α_i = 1 for all i -- i.e. the possible frequencies are uniformly distributed. Here's some code I used, which should be pretty easy to adapt to agree with the API used in this package. Dirichlet is taken from Distributions.jl.

function bayesian_bootstrap(sample::T, bootstrap_count::Integer, statistic::Function) where {T<:AbstractArray}
    data_size = length(pointwise_ev)
    weights = similar(pointwise_ev, (bootstrap_count, data_size))
    for i in 1:bootstrap_count
        weights[i, :] .= rand(Dirichlet(ones(data_size)), bootstrap_count)
    return statistic.(sample, each_col(weights))
end

[juliangehring]

Nice suggestion! I'll need to do more reading on the Bayesian bootstrap - it could be a good addition to the existing sampling methods.

If I understand it correctly, this would only work with statistic functions that support weights?

[ParadaCarleton]

Nice suggestion! I'll need to do more reading on the Bayesian bootstrap - it could be a good addition to the existing sampling methods.

If I understand it correctly, this would only work with statistic functions that support weights?

Correct -- the Bayesian bootstrap works by assuming the following:

1. The distribution of possible draws is supported only on the observed data points. (An extremely strong assumption, but one shared by the normal bootstrap. The Bayesian bootstrap was initially introduced as a way of showing that the bootstrap makes much stronger assumptions than you'd initially expect.)
2. Because we have no information except that all observed data points are possible, we should assume that every observation is equally likely and choose maximum entropy distribution over the observed data points. This means we draw PDFs from a uniform Dirichlet distribution. (An intuitive way of thinking of this is that you break a stick of length 1 into n pieces, completely at random -- any partition of that stick is equally likely. The length of each piece is the frequency we assign to each observation.)

This gives results similar to the bootstrap in practice, but much smoother for small samples -- if our sample is -1, 1 the support of the bootstrap mean is the set of points (-1, 0, 1) while the Bayesian bootstrap will generate a bootstrap distribution with support on the full range from -1 to 1.