Pathwise sampling

[rossviljoen]

It would be nice to eventually add pathwise posterior sampling from [1].

Essentially, reimplement https://github.com/aterenin/SparseGaussianProcesses.jl but using AbstractGPs etc.

[1] Wilson, James, et al. "Efficiently sampling functions from Gaussian process posteriors." International Conference on Machine Learning. PMLR, 2020. https://arxiv.org/abs/2002.09309

[willtebbutt]

Yeah. I spoke with Alex the other day. We could rope him in to this at some point if that would be helpful.

[theogf]

I already implemented it in another project, I could also help

[rossviljoen]

I've implemented a basic version of this here.

There are essentially two parts to pathwise sampling:

1. An approximate sample from the prior GP (usually by using random fourier features)
2. A posterior update using the posterior approximation (i.e. SVGP)

Since random fourier features (RFF) is a general method for approximating stationary kernels, it seems like it would make sense to put that in a separate package (possibly in KernelFunctions itself?) and just include 2. here. I will initially just make a PR to move it all into this repo for now though.

[theogf]

This sounds great, for the RFF, I would not like to clutter KernelFunctions even more. Maybe we could have an additional package which deals strictly with the whole Fourier representation of kernels.