bayes-skopt icon indicating copy to clipboard operation
bayes-skopt copied to clipboard

Augment hyperposterior sampling by also sampling observed values

Open kiudee opened this issue 2 years ago • 0 comments

In the current implementation, we sample the hyperparameters of the Gaussian process and average across those samples. The training data is fixed. In BoTorch, the hyperparameters are fixed and the observations y of the data points are sampled, and the acquisition function is averaged over those.

I think both ideas can be combined, allowing the user to request:

  • Holding both hyperparameters and observations fixed: This will yield the classical acquisition functions for the noiseless case.
  • Sampling only the observations y. This will yield the noisy acquisition function versions, assuming that the hyperparameters are accurate.
  • Sampling only the hyperparameters. This will yield the current behavior, where we are robust to model misspecification.
  • Sampling both the hyperparameters and the observations. Combining the strengths of both approaches.

The steps required are:

  • Implement a fast method for cloning the GP model using the same, or different hyperparameters.
  • Add the different methods for averaging to https://github.com/kiudee/bayes-skopt/blob/8f1daf996e34b95af47ef0d382d57fe8a17bbae5/bask/acquisition.py#L48

kiudee avatar Oct 12 '21 08:10 kiudee