stan-dev
Improve docs for variational inference
Summary:
I would like to use the ADVI implemented in Stan but can't find helpful docs.
Description:
My model needs to fit a distribution based on KL divergence. This is possible with variational inference and as far as I understand, Stan has support for VI with it's ADVI functionality.
However, the user guide and reference manual contain only very superficial information on how VI is implemented internally. They don't provide instructions on how to use it.
It would be very helpful for me to see a working toy example. Let's say I have a Poisson distribution with gamma=10, and would like to represent it with a Gaussian which minimizes KL divergence.
Additional Information:
I've asked an equivalent question on SO: https://stackoverflow.com/questions/59276428/stan-how-to-use-variational-inference-to-fit-a-distribution
Current Version:
v2.21.0
hi @lhk (Lars), agreed, the docs are mighty thin. the CmdStanPy docs have roughly the same information, but with a notebook and perhaps you'll find this useful - https://cmdstanpy.readthedocs.io/en/latest/variational_bayes.html https://github.com/stan-dev/cmdstanpy/blob/master/docs/notebooks/Variational%20Inference.ipynb
Which interface are you using? Those docs (the user's guide and reference manual) try to not be interface specific I think.
Rstan has the 'vb' call which should work similarly to the 'sampling' call used for mcmc.
Modifying the optimization example from the Rstan vignette (https://cran.r-project.org/web/packages/rstan/vignettes/rstan.html#optimization-in-stan), if you have the file model.stan:
data {
int<lower=1> N;
real y[N];
}
parameters {
real mu;
}
model {
target += normal_lpdf(y | mu, 1);
}
Build and run ADVI with:
sm <- stan_model("model.stan")
y2 <- rnorm(20)
fit <- vb(sm, data = list(y = y2, N = length(y2)))
PyStan has a vb call as well (example after the definition here: https://pystan.readthedocs.io/en/latest/api.html#pystan.StanModel.vb)
The Stan discourse is probably a better place to get answers for questions like this: https://discourse.mc-stan.org/
