Support dependencies between optimization parameters

[Engineero]

It would be great if we could enter dependencies between parameters for optimization. For example, if I have the parameter bounds:

pbounds = {'a': (0, 5),
           'b': (0, 10)}

and know that for my application, no meaningful results will come from parameter choices where b < a, I would like to be able to force the optimizer to only choose samples such that b >= a.

See [M. T. M. Emmerich et. al. 2008] for some discussion of this. That's just about the only paper that tries it that I can find so far.

[Engineero]

See pull request #95

[fmfn]

I'll take a look at the PR. But one idea that sometimes can be used is to parametrize your parameters differently, making the dependency more explicit:

instead of pbounds = {'a': (0, 5), 'b': (0, 10} with the condition b < a, you could change coordinates to: x = a and y = b - a, and define pbounds = {'x': (0, 5), 'y': (0, 5)}, and call your function like so: f(a=x, b=x + y).

[meyermj]

I have a similar requirement, but different in a key way:

pbounds = {'a': (0, 1),
           'b': (0, 1),
           'c': (0, 1)}

where a+b+c=1. I could be overlooking something, but I don't think there's a way to change my parameterization/function call to enforce this condition. Any pointers would be greatly appreciated!

[dkhachatrian]

@meyermj I think for your setup, you can solve it by recasting in terms of spherical coordinates.

First, since $a,b,c > 0$, there exists a one-to-one correspondence $x = \sqrt{a}, y = \sqrt{b}, z = \sqrt{c}$ so you can rewrite as $x^2 + y^2 + z^2 = 1$.

Now, convert to spherical coordinates, via something like:

from math import atan2, sin, cos

#...

r = (x^2 + y^2 + z^2)**(1/2)
theta = atan2(y,x)
phi = atan2(z,(x^2 + y^2)**(1/2))

Now in spherical coordinates, your pbounds can be written as

# "a+b+c = 1" -> r must be 1 -- just need to search over angles
pbounds = {'theta': (0, math.pi/2), 'phi': (0, math.pi/2)} 

and you can convert back to $x,y,z$ via

z = r*cos(phi)
y = r*sin(phi)*sin(theta)
x = r*sin(phi)*cos(theta)

then take square roots to get your original $a,b,c$.

(This can almost certainly be extended for more than three dimensions $a_1^2 + a_2^2 + ... + a_n^2 = 1$, but I don't know off the top of my head/haven't derived the relevant n-dimensional hyperspherical conversions haha.)

[hammer-wang]

I'll take a look at the PR. But one idea that sometimes can be used is to parametrize your parameters differently, making the dependency more explicit:

instead of pbounds = {'a': (0, 5), 'b': (0, 10} with the condition b < a, you could change coordinates to: x = a and y = b - a, and define pbounds = {'x': (0, 5), 'y': (0, 5)}, and call your function like so: f(a=x, b=x + y).

Thanks! This is awesome.

[bwheelz36]

addressed in #344