Add FOBO Tutorial

[yyexela]

Motivation

Add tutorial utilizing dKG and dNEI for dGP functionality.

Have you read the Contributing Guidelines on pull requests?

Yes

Test Plan

N/A (notebook runs without issues).

Related PRs

https://github.com/pytorch/botorch/issues/1679

[Balandat]

Where can I see the versions of the packages installed for the smoke test? I was able to run my notebook just fine locally with a fresh conda environment and latest package versions, but got an error in my PR...

The job is defined here: https://github.com/pytorch/botorch/blob/main/.github/workflows/tutorials_smoke_test_on_pr.yml#L10-L17, so you'll see that this will run against the latest changes on the main / primary branch of pytorch/gpytorch/botorch: https://github.com/pytorch/botorch/blob/main/.github/workflows/reusable_tutorials.yml#L53-L55

Can you see the logs here: https://github.com/pytorch/botorch/actions/runs/7079121747/job/19265397371?pr=2137 ? The setup stages show the logs including which packages are installed (assuming there isn't any permissions issue with viewing the full logs).

[Balandat]

I wonder if this could be related to some of @sdaulton's recent changes, e.g. https://github.com/pytorch/botorch/pull/2011 ?

[yyexela]

That doesn't appear to be the case for me, it looks like it's a result of this PR from GPyTorch (https://github.com/cornellius-gp/gpytorch/pull/2288) which changes the shapes of some objects (in particular, mean_cache, which used to be size [5,1,1,44] to [5,5,5,44]).

My code runs on the commit just before that one, using the latest BoTorch commit, but breaks when using the commit from the PR (https://github.com/cornellius-gp/gpytorch/pull/2288).

[yyexela]

This change in particular is where I think the issue is: https://github.com/cornellius-gp/gpytorch/commit/981edd83a671e8dca9d67c91b188702354884a34#diff-2c80c12a0a4206c9d43cd7be2a12f5c8709a4229fcf28eea27836078b0bc24f9L306-R342

[yyexela]

Ok, so I got it working locally, I changed

mean_cache = mean_cache.squeeze(1)

to

mean_cache = mean_cache.permute(2,0,1,3)[:,0:1,0,:]

and it works as before. This extracts the relevant dimensions to match what was previously working. I verified heuristically that the dimensions I got rid of were copies of one another, so we keep the relevant data. I'm not sure why caused the shape of mean_cache went from [5,1,1,4] to [5,5,5,4], in the first place (maybe from https://github.com/pytorch/botorch/pull/2011, like you mentioned ?), but I just put in a PR in GPyTorch for this change for now. Hopefully those tests pass... lol

EDIT: Unit tests aren't passing but neither are the ones in main for GPyTorch, so I'll assume this change is what's needed for now. (link: https://github.com/cornellius-gp/gpytorch/pull/2452)

[yyexela]

Any updates on how to get this through? I had someone reach out to me today because they ran into the same issue (https://github.com/yyexela/botorch/commit/a39e6f51fdaaa8b8760717a83b23f6678c61473f#commitcomment-137448122). I believe all we need is for (https://github.com/cornellius-gp/gpytorch/pull/2452) to merge.

Thanks!

[yyexela]

@Balandat

[saitcakmak]

Hi @yyexela . Thanks for the tutorial! I requested a review on the GPyTorch PR, hopefully someone will get to it soon. We're trying out a new support model for community contributions: https://github.com/pytorch/botorch/blob/main/CONTRIBUTING.md#community-contributions-beta. Once the GPyTorch fix lands, can you move the tutorial to the notebooks_community directory and add the contributor info and other metadata as outlined in the readme file? Thanks!