Particle marginal MH

[reubenharry]

The following are some notes on PMMH, which is an inference algorithm that it might be interesting to implement using the asynchronous FRP tools of Rhine. In particular, this would be an online version of the PMMH algorithm.

I use a >--> b to mean forall m . MonadDistribution m => BehaviourF m Double a b

The setting is that we have a transition model tm :: Theta >--> Z, that expresses how parameters theta influence hidden states, and an observation model om :: Z >--> Y which expresses how hidden states result in observations. Moreover, we have a prior pr :: () >--> Theta = constM pTheta, for some distribution pTheta. That is, theta is stationary. The naive approach of

particleFilter proc data -> do 
  
  theta <- pr -< ()
  latent <- tm -< theta
  arrM factor -< the_likelihood latent data
  returnA -< (latent, theta)

will be, while asymptotically correct, useless, because the values of theta for all particles will never change from their initial state.

What we would like to do instead is run an MCMC chain over theta. To do this exactly, we would need to calculate $p(y,\theta)$ and use it in the MH ratio. However, (and this is the PMMH approach), we can instead use an unbiased estimate of $p(y,\theta)$ obtained by importance sampling or a particle filter. In particular, the output of system evidence as defined here:

evidence :: Theta >--> Double
evidence = sum . arr sumWeights . particleFilter proc theta -> do 
  
  latent <- tm -< theta
  arrM factor -< the_likelihood latent data

is (roughly) what we need.

On the other hand, MCMC can be expressed as a (discrete) system:

mcmc :: Double >--> Theta
mcmc = ...

The general idea is that these would be run on separate clocks, but with communication. Every so often, the clock for mcmc would tick, at which point a new value of Theta would be chosen. Then evidence would run for a while, until a reasonably good estimate of the evidence was calculated, good enough to take another step of mcmc.

Obviously this is just a very high level sketch.