Connect with Turing.jl

[cscherrer]

Converting a Soss model to a Turing model is simple, and Soss can generate code. Shouldn't be hard to automate this.

On the other side, let's make sure it's easy to use a Soss model as a distribution within a Turing model.

[mohdibntarek]

Hi Chad, I thought it's about time we integrate Soss and Turing! So I think the easiest path forward is to:

1. Define a function that converts a Soss model to a distribution. You can have a SossDistribution struct that wraps a Soss model. This should define rand and logpdf and subtype Distribution.
2. Define Bijectors.logpdf_with_trans, Bijectors.link and Bijectors.invlink on SossDistribution, where link takes us from the constrained space to the Euclidean space, invlink is the inverse. logpdf_with_trans can probably be defined automatically using logabsdetjac and Bijectors.Stacked (@torfjelde ?)
3. Define DynamicPPL.vectorize and DynamicPPL.reconstruct for SossDistribution to vectorize and reconstruct the named tuple.

After these 3 steps, you should be able to embed a Soss model in a Turing model as a distribution.

CC: @yebai @xukai @trappmartin @devmotion

[cscherrer]

Hi @mohamed82008 ,

Sorry I had missed this! I'd love to connect Soss with Turing.

I already have rand and logpdf defined for any model. Just to be clear, you need the subtyping as well? I've been planning to move away from the Distributions library (it's very limiting) but connecting it for this shouldn't be too bad.

The Bijectors stuff would be easy too, since I'm already doing this with TransformVariables (and I'd love to get Bijectors going anyway). But some values (observed or not) could be discrete, which would throw this off.

This has me thinking we might still have some details left to work through...

[mohdibntarek]

So after https://github.com/TuringLang/Bijectors.jl/pull/99 is merged (soon), you can get link, invlink and logpdf_with_trans defined automatically if you define logpdf and bijector for your distribution. Subtyping Distribution is important yes unfortunately. All you need is a SossDistribution struct that wraps a Soss model. SossDistribution(m::SossModel) would construct a subtype of Distribution. What do you think?

[cscherrer]

Sounds reasonable, but it will only work for a few cases. What if some values are discrete?

[mohdibntarek]

What if some values are discrete?

Bijectors handles discrete variables no problem using the Identity bijector.

[mohdibntarek]

julia> using Bijectors

julia> bijector(Bernoulli(0.5))
Identity{0}()