Quadrature alternatives

[st--]

Feature request

We've made some strides improving the quadrature handling in GPflow. It would be amazing if we could also make it possible to use alternatives to the default Gauss-Hermite quadrature, for example Unscented Transform and Monte Carlo quadrature.

Motivation

Why does this matter? The default Gauss-Hermite quadrature scales poorly with more dimensions. Performing quadrature with D dimensions and N points per dimension results in a summation over N^D terms. This can be problematic for example when using MultiLatent Likelihoods (introduced in #1559) with more than two dimensions.

Example: A practical example would be a heteroskedastic StudentT likelihood with 3 latent GPs modelling the location, scale and degrees-of-freedom parameters.

Proposal

Describe the solution you would like

The implementation of other quadrature methods such as Monte Carlo quadrature and Unscented Transform would add great options to overcome dimensional scalability problems.

Ideally, it should be possible to alternate between quadrature methods inside the same likelihood. For example, It should be possible to, to run Gauss-Hermite quadrature after some optimization iterations computed with Unscented Transform to make sure that the approximations are in agreement.

What alternatives have you considered?

Are you willing to open a pull request? (We really appreciate contributions!)

[spectraldani]

Hi! I have been working on UT and MC sampling for the expectation module for GPflow 1.x for a while now, and I'm willing to help with quadrature code! I just need some time to get used to TF 2.x.

Ideally, it should be possible to alternate between quadrature methods inside the same likelihood. For example, It should be possible to, to run Gauss-Hermite quadrature after some optimization iterations computed with Unscented Transform to make sure that the approximations are in agreement.

I remember trying to implement like this back in GPflow 1.x, using the Strategy-like pattern I alluded at https://github.com/GPflow/GPflow/issues/1205#issuecomment-729988436, but because of graph mode, I needed to "rebuild" the model after swapping the quadratures to update the likelihood tensor. I assume this is no longer be a problem in GPflow 2.x, right?