Comparison to existing tuners

[zkurtz]

Hi this looks like a great project! I'm interested if you'd add a brief summary of what advantages this provides over existing tuners. Maybe the most closely related is mlrMBO?

[AnotherSamWilson]

I will get to work on a thorough benchmark against mlrMBO tomorrow (I haven't look too hard at the mlMBO package). I do have benchmarking available against the rBayesianOptimization package, if you are interested.

[zkurtz]

I'm not aware of a widely-accepted or standardized way of benchmarking tuners, so whatever you come up with could be of interest. Beyond benchmarking, I was also just interested in whether your algorithms work fundamentally differently (in terms of probability theory or in terms of computational tricks for speed/efficiently) compared to previous tuners.

[waiyujack]

Hi Sam, A really nice package. Hope you could keep developing it. I was wondering if you could give out some benchmark on your package? Also, is it possible to put it on Kaggle?

[AnotherSamWilson]

I am actually working on a kernel now, unfortunately my time has been spent working on improvements to the package instead of benchmarking. Here is an example of a kernel that uses ParBayesianOptimization:

https://www.kaggle.com/srwilson7601/autoencoders-bayesian-optimization-and-xgboost

I do plan on getting to a thorough comparison at some point. I have seen some initial results comparing ParBayesianOptimization to rBayesianOptimization, however I don't want to release anything without reproduceable code along with it.

[dkatztibco]

Trying to run the example in the R docs but I get this error: tNoPar <- system.time( ScoreResult <- BayesianOptimization( FUN = scoringFunction , bounds = bounds , initPoints = 4 , bulkNew = 1

, nIters = 6

, n_iter = 6
, kern = kern
, acq = acq
, kappa = 2.576
, verbose = 1
, parallel = FALSE)

) #as published..

Error in BayesianOptimization(FUN = scoringFunction, bounds = bounds, : could not find function "BayesianOptimization"

This function is not found after loading the library. I also tried using the function of the same name in package rBayesianOptimization but it doesn't seem to work. Any help appreciated.

Regards,

[AnotherSamWilson]

The API has changed after release 1.1.0 https://github.com/AnotherSamWilson/ParBayesianOptimization/releases/tag/v1.1.0

• BayesianOptimization() has been replaced with bayesOpt() and its supporting methods. See the readme https://github.com/AnotherSamWilson/ParBayesianOptimization#simple-examplefor examples.

On Sun, Jan 10, 2021 at 2:51 PM dkatztibco [email protected] wrote:

Trying to run the example in the R docs but I get this error: tNoPar <- system.time( ScoreResult <- BayesianOptimization( FUN = scoringFunction , bounds = bounds , initPoints = 4 , bulkNew = 1 , nIters = 6

, n_iter = 6 , kern = kern , acq = acq , kappa = 2.576 , verbose = 1 , parallel = FALSE)

) #as published..

Error in BayesianOptimization(FUN = scoringFunction, bounds = bounds, : could not find function "BayesianOptimization"

This function is not found after loading the library. I also tried using the function of the same name in package rBayesianOptimization but it doesn't seem to work. Any help appreciated.

Regards,

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/AnotherSamWilson/ParBayesianOptimization/issues/1#issuecomment-757533372, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKUWFOOOUS3VLZMK7KOS36TSZIAKVANCNFSM4GNAZDCA .

--

---

Samuel Wilson

[dkatztibco]

Correction, the #comment distorted the post. Here is what was meant:

Trying to run the example in the R docs but I get this error: tNoPar <- system.time( ScoreResult <- BayesianOptimization( FUN = scoringFunction , bounds = bounds , initPoints = 4 , bulkNew = 1 , nIters = 6 , kern = kern , acq = acq , kappa = 2.576 , verbose = 1 , parallel = FALSE) ) (from the docs)

Error in BayesianOptimization(FUN = scoringFunction, bounds = bounds, : could not find function "BayesianOptimization"

This function is not found after loading the library. I also tried using the function of the same name in package rBayesianOptimization but it doesn't seem to work. Any help appreciated.

Regards,