Parallelisation

[lesshaste]

It would be great if the optimisation could take advantage of multiple cores

Describe the solution you'd like

There is a method described in https://arxiv.org/pdf/1206.2944 section 3.3 which is already implemented by the R package https://cran.r-project.org/web/packages/ParBayesianOptimization/ParBayesianOptimization.pdf .

[till-m]

Hey @lesshaste,

I think the paper you sent is already outdated. I looked into this a year or two ago, and it seemed that the best approaches at the time were locally-penalized batches, however I opted for implementing the ConstantLiar from Ginsbourger et al. (which may already be sufficient for your purposes).

In any case before implementing anything else it would be good to check the current state of research for a review paper.

[lesshaste]

https://www.sciencedirect.com/science/article/pii/S0098135422004434 contains what looks like a good overview of available methods.

Section 3.9 of https://dl.acm.org/doi/full/10.1145/3582078 also has a good discussion.

[jacktang]

@lesshaste do you consider GPU?

[lesshaste]

I wasn't thinking about GPUs.

I have since done some more reading about my question. The term I should have used is batch bayesian optimisation. It looks like GIBBON is the cutting edge bayesian optimisation method in general. It is in botorch for example and it supports batch bayesian optimisation. https://jmlr.org/papers/volume22/21-0120/21-0120.pdf