Kernel vs GP composition

[kaandocal]

Hi there, I am trying to get into Stheno a little but I'm having a little trouble figuring out whether there are any notable differences between some transformations applied to GPs vs. kernels and mean functions. Examples are stretch vs. KernelFunctions.ScaleTransform, or GP + GP vs. Kernel + Kernel and Mean + Mean (I assume this would be useful when combined different likelihood functions). Would there be any loss eg. in defining stretch to work via input transforms on the kernel and mean functions?

[willtebbutt]

Hi! Glad to hear you're trying the two approaches out.

There's no issue per-se in taking either approach. Certainly, you can recover the process that you get by writing GP + GP using Kernel + Kernel and Mean + Mean, for example. The distinction is whether or not you're able to decompose the posterior into its components.

For example, writing something like

f = @gppp begin
    f1 = GP(SEKernel())
    f2 = GP(Matern12Kernel())
    f3 = f1 + f2
end

lets you write things like

f_post = posterior(f(GPPPInput(:f3, x), y)
f_post(GPPPInput(:f2, x))

to get the posterior over f2 given y at some locations x, and you could do the same for f1. If you write Kernel + Kernel, it's not straightforward to get at anything other than the equivalent of the posterior over f3.

Does this help?

[kaandocal]

Thanks a lot, that's very helpful! In any case being able to add GPs is great functionality. How about stretch?

[willtebbutt]

It's a similar story with stretch :)

f = @gppp begin
    f1 = GP(SEKernel())
    f2 = stretch(f1, 0.5)
end

f2 should be equivalent to

GP(SEKernel() ∘ ScaleTransform(0.5))

(I might be off by a constant factor)

It's probably easier to get at f1 via f than it would be to get at it manually via the GP, if for some reason that's what you wanted to do, but it's less of a clear win than in the case of addition of processes.

[kaandocal]

That makes sense. I too am less convinced by this example, but thanks very much for the clarification!

[willtebbutt]

Haha glad you agree.

I'm keen to get differentiation / integration working in addition to addition etc, but haven't had the time. These are the kinds of things that I think would also be very convincing, and potentialy have quite a lot of utility.

[kaandocal]

These would definitely be great ideas and would look great in any demo.