Integrate code from GPyOpt: addresses several issues

[simonmb]

I wanted to propose to adapt some of the code from aproject which is not maintained anymore: GPyOpt. With code from this project you could address issues #308, #191, #93, #94, #189 relating to mixed float and integer variables and defining equality, inequality and even function based constraints on the bayesian optimization. I am aware of pull request #95, but the way how it is implemented in GPyOpt is more robust, based on Gardner et al., 2014.

[bwheelz36]

I think especially a robust way to handle constrained optimization would be excellent; it's been a repeated request so it would be great if it it could be solved in an elegant and robust way. I guess from my side, I would want to see the usual things: original tests passing, new tests written to cover the new code, doc strings, and examples.

[tsbertalan]

This is a nice idea. The BSD 3-clause license of GPyOpt seems pretty permissive (although it might be a good gesture to contact the authors and let them know). This package seems to have superseded GPyOpt in search results in the past year, so it would be really good for it to have feature parity.

For myself, handling both integer parameters and constraints (possibly black-box nonlinear constraints) would be very useful features, though I'm aware of the suggested workarounds.

[till-m]

@simonmb do you know how GPyOpt handles integer parameters and whether it's different from the paper by Garrido-Merchán and Hernández-Lobato brought up in #191?

As for constraints, it might be easier to implement Gardner et al. from scratch rather than adapting the GPyOpt code.

[simonmb]

No, I am sorry. I have not had a look in detail on the features nor on their implementation. I just wanted to put it out there, that maybe some features could be implemented here as this package is still being maintained.