Add capability to write custom sampler

[ashiklom]

I am interested in functionality that would allow users to combine different samplers using different parameter subsets. My specific use case is that some of my parameters have a conjugate likelihood - prior combination, meaning I could much more efficiently sample that subset of parameters with Gibbs sampling (i.e. 100% success rate).

More generally, this could require a substantial refactoring of the code that separates the MCMC sampling loop -- which is broadly general across all samplers -- from the sampling step that happens at each iteration. I.e. the MCMC loop (for all MCMC-based samplers, not SMC) could look something like:

for (i in 1:niter) {
    params[i, subset_DE] <- sample_DEzs(params[i-1, subset_DE], settings$DEzs)
    params[i, subset_Metropolis] <- sample_Metropolis(params[i -1, subset_Metropolis], settings$Metropolis)
    # User-defined sampler. For instance, a Gibbs sampler for conjugate parameters
    params[i, sample_userdef] <- sample_userdef(params[i - 1, subset_userdef], settings$userdef)
}

Does this sound like something that is at least remotely feasible? I could definitely contribute to helping with implementation, or potentially even take the lead on it in a separate branch, but I wanted to see if this is something you had considered before and if there are underlying features of the samplers that make this impossible.

[florianhartig]

Hi Alexey,

hmmm ... sounds remotely feasible, but my feeling is that we would need some good thinking about the structure to avoid making this a maintenance nightmare.

I've been wondering about this sort of thing and have actually been pondering a re-structuring of the code so that there is only one MCMC sampler, which can then be given different proposal functions to implement the different samplers that we currently have.

Once such a structure is in place, it would be relatively easy to provide a message to the acceptance to account for conjugacy.

I'll discuss this with my co-developers, and you'd be very welcome to help if you have the time. However, I guess we would only do this for the 0.2 release, in some time, so if you need this soon, I'd say go ahead and do this by hand.