Modeling Derivatives et al

[willtebbutt]

I'm opening this to scope out + plan the best manner in which Stheno.jl can be extended (or, rather, re-extended -- we used to have functionality that did this but it had some serious limitations) model the derivatives and related quantities (gradients, hessians, jacobians, etc) of functions.

There are two main things to consider:

1. what is the scope of the things that we want to handle, in particular do we want to cover both multiple inputs and higher-order derivatives in an initial pass?
2. how to actually implement these things. For example, we've recently added support for multi-output GPs in KernelFunctions.jl, which might be helpful for e.g. modeling the gradient of a GP. I am very keen to avoid re-inventing the wheel, so perhaps a pre-requisite is to get Stheno.jl using KernelFunctions.jl?

[emsal0]

How much of the backend would have to be refactored if we start using KernelFunctions.jl in Stheno.jl?

[willtebbutt]

Actually not too much at all -- it's more that there's one thing I need to add to KernelFunctions.jl to make it possible to refactor stuff.

Using KernelFunctions should result in a net loss of code, which is cool.

[emsal0]

It looks like then that the KernelFunctions route would probably be optimal, assuming that that one thing is easy enough to do - imo it pays to be patient and avoid incurring tech debt by forging ahead and having to heavily refactor things later.

[willtebbutt]

Sounds good to me. I'm hoping to have the transition complete in a few day's time anyway, so not too long to wait.

[emsal0]

Checking up on this, how's progress on the KernelFunctions thing?

[willtebbutt]

Hiya. It's taking longer then I had expected because other commitments are getting in the way of resolving the issues remaining in #137. I should be able to take another look at it once the AABI deadline has passed.

[emsal0]

No problem, I've been quite busy as well so my time scale for this is really quite flexible