gpytorch icon indicating copy to clipboard operation
gpytorch copied to clipboard

add combine_terms option to exact MLL

Open samuelstanton opened this issue 3 years ago • 4 comments

I've found that logging the inv_quad terms and logdet terms separately (rather than just the train loss) to be very helpful for debugging. Right now classes like VariationalELBO have a combine_terms option that allow the user to sum the terms after the MLL call. This is a nice feature, since otherwise you essentially have to pay for an extra training step just to log the terms separately.

In this PR I've demonstrated how we could go about adding this option to the subclasses of MarginalLogLikelihood, starting with the Gaussian likelihood case. There are a few unit tests that aren't passing yet, but I wanted to check and see if this feature would be approved before fixing it up.

samuelstanton avatar Dec 16 '21 18:12 samuelstanton

@jacobrgardner @gpleiss any thoughts?

samuelstanton avatar Dec 18 '21 17:12 samuelstanton

Yeah, this would be awesome to add!

gpleiss avatar Dec 21 '21 01:12 gpleiss

@gpleiss how does everything look?

samuelstanton avatar Dec 23 '21 18:12 samuelstanton

Looks like the failing unit test was flaky.

wjmaddox avatar Feb 03 '22 16:02 wjmaddox