Probability interface tutorial

[ParadaCarleton]

First addition to the DynamicPPL tutorials; breaking this up as Hong suggested. Goes over how to use the basic interfaces (e.g. logjoint, loglikelihood, logdensityof).

[yebai]

Many thanks @ParadaCarleton - looks like a good PR!

[yebai]

bors r+

[devmotion]

It seems it was not tested and is not included in the documentation?

[yebai]

It seems it was not tested and is not included in the documentation?

Sure, the plan is to have a series of small PRs to replace #365 - it is probably ok to add testing in a subsequent PR, but if @ParadaCarleton can quickly add a fix that would be good too.

[devmotion]

Yeah, that would be good. It's apparently also a jmd file but we don't support these here in DynamicPPL yet.

[devmotion]

bors r-

[bors[bot]]

Canceled.

[ParadaCarleton]

@yebai should be ready to merge

[yebai]

bors r+

[yebai]

bors r-

[bors[bot]]

Canceled.

[yebai]

bors r+

[bors[bot]]

:-1: Rejected by code reviews

[yebai]

bors r+

[bors[bot]]

:-1: Rejected by code reviews

[yebai]

@devmotion, it seems bors is not happy for me to contribute and approve this PR. Can you approve and merge it?

[yebai]

@devmotion can you take another look? We would like to move this forward and can perform additional improvements even after merged.

[devmotion]

The main issue is that, as mentioned above, currently the cross-validation example does not work: https://github.com/TuringLang/DynamicPPL.jl/actions/runs/3726471522/jobs/6319942583#step:5:248

Even some other example fail (e.g., https://github.com/TuringLang/DynamicPPL.jl/actions/runs/3726471522/jobs/6319942583#step:5:231) which, however, can be fixed easily by making FillArrays a direct dependency and loading it.

Probably it would be good to make the docs build fail if there are any errors in the examples.

The errors can be deduced also from the missing output in https://beta.turing.ml/DynamicPPL.jl/previews/PR404/tutorials/prob-interface/.

[devmotion]

I would suggest the following:

• Change the model to a Gaussian model with normal-inverse gamma prior
• Write a function bayes_loss(dataset)
• Use samples from the exact posterior instead of samples from NUTS

Then the example can be executed without depending on Turing and without having to deal with Chains or StructArray. IMO that will make the example more readable, I imagine it could be something like:

function bayes_loss(dataset::Vector{Float64}; nsamples=1_000, nfolds=5)
    posterior_samples_mu = Vector{Float64}(undef, nsamples)
    posterior_samples_sigma2 = Vector{Float64}(undef, nsamples)
    loss = 0.0
    for (training, validation) in kfolds(dataset, nfolds)
        # Sample from the exact posterior
        rand!(InverseGamma(...), posterior_samples_sigma2)
        rand!(MvNormal(..., posterior_samples_sigma2 ./ ...), posterior_samples_mu)

        # Estimate Bayes loss
        model = gdemo(length(validation)) | (x=validation,)
        loss += sum(1:nsamples) do i
            logjoint(model, (mu=posterior_samples_mu[i], sigma2=posterior_samples_sigma2[i]))
        end
    end
    return loss
end

[ParadaCarleton]

bors r+

[bors[bot]]

Pull request successfully merged into master.

Build succeeded:

• docs
• format
• test (1.6, ubuntu-latest, x64, 1)
• test (1.6, ubuntu-latest, x64, 2)
• test (1, macOS-latest, x64, 2)
• test (1, ubuntu-latest, x64, 1)
• test (1, ubuntu-latest, x64, 2)
• test (1, ubuntu-latest, x86, 2)
• test (1, windows-latest, x64, 2)
• Turing.jl

[yebai]

thanks @ParadaCarleton and @devmotion!

[YongchaoHuang]

I would suggest the following:

* Change the model to a Gaussian model with normal-inverse gamma prior

* Write a function `bayes_loss(dataset)`

* Use samples from the exact posterior instead of samples from NUTS

Then the example can be executed without depending on Turing and without having to deal with Chains or StructArray. IMO that will make the example more readable, I imagine it could be something like:

function bayes_loss(dataset::Vector{Float64}; nsamples=1_000, nfolds=5)
    posterior_samples_mu = Vector{Float64}(undef, nsamples)
    posterior_samples_sigma2 = Vector{Float64}(undef, nsamples)
    loss = 0.0
    for (training, validation) in kfolds(dataset, nfolds)
        # Sample from the exact posterior
        rand!(InverseGamma(...), posterior_samples_sigma2)
        rand!(MvNormal(..., posterior_samples_sigma2 ./ ...), posterior_samples_mu)

        # Estimate Bayes loss
        model = gdemo(length(validation)) | (x=validation,)
        loss += sum(1:nsamples) do i
            logjoint(model, (mu=posterior_samples_mu[i], sigma2=posterior_samples_sigma2[i]))
        end
    end
    return loss
end

@devmotion Just curious about the above code block, maybe two stupid questions:

1. is the logjoint accepting (model, NamedTuple)?
2. are we calculating loss or just log likelihood here? if it's loss, shall we return -loss?