Venturecxx
Venturecxx copied to clipboard
probcomp
Should enumerative Gibbs fix the randomness with which non-principal consequences are simulated?
By analogy with HMC and Slice (both of which were implemented considerably later). #208 seems relevant.
No problems with failing to fix the randomness have been detected to date, but there is also no recorded proof that not fixing it is fine, and I, for one, have self-doubt about my intuition here without going through the formal exercise.
Brainstorming a possible direction for an example: Something with multiple options generating brush that really ought to go the same way each time?
Or is fixing randomness not required, because the full trace proposed is the same as the candidate generated when evaluating that log density?
My guess is that fixing randomness is not required, basically because of Neal's Algorithm 8 (i.e. you need to fix the randomness for at least the one case that corresponds to the current state, which was #208, but if you do that you're fine).
I think it's also not required for slice; a similar-sounding assertion is made in http://arxiv.org/pdf/1510.02958v1.pdf, section 3, although a detailed justification of this for our case could be useful.
That said, fixing randomness seems valid too, and might be desirable, so maybe we want to expose that as a choice.
Conjecture: fixing randomness is a particular way of fixing some deterministic function from the principal values to the brush values, and other ways might also work (as long as they produce the correct distribution on the brush). In particular, the method of "regenerate the brush with fresh randomness, but only if we haven't seen the same set of principal values before" would be equivalent to what enumerative Gibbs does (and possibly what slice does, although it may still suffer from an analogue of #208).
Hm. You may be right. As I recall, the argument for why one might want fixed randomness for HMC was always aesthetic, and we never did produce a proof that not fixing it wouldn't work.