DistributionsAD.jl
DistributionsAD.jl copied to clipboard
Markovian distribution using Zygote
Currently, time-series models are very slow in Turing compared to Stan mainly because of the dynamic dispatch of Tracker.jl. Loops are hard to avoid in such models and loops currently make Tracker AD super slow. Last I checked, Zygote was also not fully ready for action to be adopted in Turing.jl. However, if we can define a Markovian distribution with a recursive function that given the previous state returns a distribution for the next state, and then make Zygote differentiate it successfully, we can then use Zygote to define a Tracker primitive for the logpdf function of this distribution much like how we handle MvNormal. This should enable fast reverse-mode AD of time series models in Turing.jl. Not sure how feasible this is, but might be worth investigating.