More optimisation methods for FORM

[AnderGray]

Might be nice to have more methods than HLRF for FORM.

This paper (although old) describes several constrained optimisation methods for FORM: https://doi.org/10.1016/0167-4730(91)90041-7

When we have GPs, can we use them to find design points with Bayesian Optimisation? https://arxiv.org/pdf/1403.4890

[teobros]

I think that there are 2 things that can be done:

1. Add iHLRF (more stable than standard HLRF and will give us feature parity with UqLab)
2. Define the design point identification as an optimization problem ( min ||Z||^2 such that h(Z) <= 0 ) and allow the user to pass an optimizer of their choice (connected to what @FriesischScott wants to do with the optimization, i.e., allow the user to use their licenses jump stuff)

Then, we can provide our own stochastic optimizers once we have them, e.g. bayesian optimization.

I only wonder one thing: is FORM supporting imprecise inputs? I think it should be a nice test-bed for the IP framework if we get an interval of beta / 2 DPs as an output.

[FriesischScott]

Currently FORM doesn't do imprecise but it should be relatively easy to adapt. Is it as easy as running it twice and using different bounds output from the models?

[AnderGray]

Would we only let the user pass constrained optimisers? Or do we somehow need to convert it to an unconstrained one (i.e. like a Lagrangian thing)

Yes, FORM is working for imprecise inputs right? In both the DL and random slicing I believe

[teobros]

I would say the user should be kept free to pass whatever optimizer they prefer, the only thing we do is construct the optimization problem in the SNS