Stheno.jl
Stheno.jl copied to clipboard
JuliaGaussianProcesses
Kronecker-factored matrices
Consider a kernel of the form
(k)(x::Vector, x′::Vector) = prod(map(k.ks, x, x′))
where k.ks is a collection of kernels which can be applied to single-dimensions, and the dimensionality of x and x′, and the length of k.ks is D.
If we wish to evaluate the covariance matrix associated with a matrix of inputs which can be represented as a Cartesian product:
X = IterTools.product(x1, ..., xD)
where each xd is a vector, then cov(k, X) can be represented as the Kronecker product of a set of D matrices, the dth of which is cov(k.ks[d], xd). If this structure is present then the asymptotic complexity of inference is greatly reduced.
- Details of this can be found in [1].
- There was work at NIPS 2017 in the AABI workshop which maybe relates to this work - see [2].
- There is no obvious reason why one shouldn't be able to exploit matrices which admit partial Kronecker factorisations. I'm not sure what / if there is relevant work is here though.
- Perhaps there is something in the tensor decomposition / tensor train field / line-of-work (I'm not even sure what to call this) that can speak to the above. This video looks related: https://www.youtube.com/watch?v=CrSFne4h3Pk
To achieve efficient inference here, it will be necessary to implement a Kronecker-factorised matrix type which has appropriate efficient representations.
[1] - Saatçi, Yunus. Scalable inference for structured Gaussian process models. Diss. University of Cambridge, 2012. [2] - Evans, Trefor W., and Prasanth B. Nair. "Scalable Gaussian Processes with Grid-Structured Eigenfunctions (GP-GRIEF)."
This is a large body of research using Kronecker factorizations in GP/kernel ridge regression, e.g. GPatt and Kronecker ridge regression. Would love to discuss this or contribute!
Would be great to both discuss and collaborate :) Are you around for the hackathon tomorrow?
Yes! I am here the whole Hackathon. Would like to get involved 🙂
On Thu, 25 Jul 2019, 16:48 willtebbutt, [email protected] wrote:
Would be great to both discuss and collaborate :) Are you around for the hackathon tomorrow?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/willtebbutt/Stheno.jl/issues/6?email_source=notifications&email_token=ABN5PI4ZC6KAWGTQW4A4SD3QBIGQXA5CNFSM4E5OIFTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD22XL4Q#issuecomment-515208690, or mute the thread https://github.com/notifications/unsubscribe-auth/ABN5PI5BN66TCIGXVLFMFB3QBIGQXANCNFSM4E5OIFTA .