Ax Troubleshooting MOO points focused in small region of search space (via looking at acquisition function values)

I'm working on a multi-objective problem and I'd like to render acquisition plots in each iteration. I looked at available plotting utils but doesnt seem there is any for my purpose. Is there an easy way to implement these plots or is it something that will be released in the next versions?

Feb 07 '22 18:02 du-210

Hi @du-210! Could you provide a bit more detail for the type of plot you are looking for? What would be the data being visualized?

Feb 08 '22 15:02 lena-kashtelyan

Hi @lena-kashtelyan, I am trying to optimise a function that I know roughly how the optimal pareto front should look like. But when I run MOO, after the first few optimisation iterations, nearly all generated points concentrated on a small region of the optimal pareto front. I need to understand why this happens. I am able to diagnose my surrogate models using Ax plotting utils but I'm in dark when it comes to acquisition function. Note that I am quite new to the world of MOO so if you think the ACF plot is not what I need to look into, please advice me to a right direction. Thanks

Feb 11 '22 16:02 du-210

Hi @du-210, sorry for the delay on this. My understanding was that you would like to know how to approach the fact that the first few trials in your MOO optimization end up concentrated in a small region of the search space, correct?

I think looking into model fit (which you've already been doing with the help of the plot) and into the values of the acquisition function for the generated trials seems reasonable. Unfortunately, doing the latter is a bit non-trivial currently, but next stable version release of Ax will have a more convenient interface for it. When that version is released (which should be towards the end of this week), we'll provide you with an example!

However, I think also just looking at the acquisition function values might not be enough –– if Ax is picking those points in the small region, likely you will just see that those points have high acquisition function values, but it won't necessarily be clear why that is, I think? To that effect, I suggest you provide a more detailed repro so we can look into this for you. For a full repro:

Please show how you are setting up your experiment, what the search space is, and approximately what values your metrics take on.
Also paste the dataframe for your experiment (output of ax.service.utils.report_utils.exp_to_df(ax_client.experiment) too.

Mar 01 '22 16:03 lena-kashtelyan

Hi @lena-kashtelyan, thanks for your detailed response! I'd like to know the reason why I don't get a good coverage of the pareto front (not necessarily concentration of point in a small region of the search space). As you can see in the plot below most points are concentrated at the bottom left corner of the pareto front. I know by changing the reference point, I can change the concentration of points but I work with a set of problems that I don't have any idea about the range of objectives and I should only rely on inferred ref points.

experiment setup:

generation_strategy = GenerationStrategy(
    name='Sobol+MOO',
    steps=[
        GenerationStep(
            model=Models.SOBOL,
            num_trials=sobol_trials,  
        ),
        GenerationStep(
            model=Models.MOO,
            num_trials=-1,  
            max_parallelism=3,
        ),
    ]
)


ax_client = AxClient(generation_strategy=generation_strategy, verbose_logging=False)

params = [
    {"name": x0, "type": "range",  "value_type": "float", "value": [-2.0, -0.1]},
    {"name": x1, "type": "range",  "value_type": "float", "value": [-4.0, 4.0]},
    {"name": x2, "type": "range",  "value_type": "float", "value": [0.1, 2.0],
    {"name": x3, "type": "range",  "value_type": "float", "value": [-4.0, 4.0],
        ]
# f0 ~ [69, 98], f1 ~ [1.2, 5]
objs = {f0: (ObjectiveProperties(minimize=False), f1: ObjectiveProperties(minimize=True))}
ax_client.create_experiment(name="test_experiment", parameters=params, objectives=objs, overwrite_existing_experiment=True, is_test=False)

the dataframe for my experiment:

,f0,f1,trial_index,arm_name,x0,x1,x2,x3,trial_status,generation_method
0,72.12,1.53647,0,0_0,-0.6470869541168214,2.5707240104675293,1.9394968152046204,2.579373359680176,COMPLETED,Sobol
3,85.79,2.0594,1,1_0,-0.6839177334681155,-2.0156011655926704,0.7604674097150563,0.8926023468375206,COMPLETED,Sobol
15,87.49,2.32283,2,2_0,-1.13101015817374,-1.0271859839558601,1.328472756035626,1.7233766093850136,COMPLETED,Sobol
27,88.29,1.93688,3,3_0,-1.2270060538314285,-2.4318535700440407,1.9887696236371994,-0.5231098160147667,COMPLETED,Sobol
39,69.79,1.39541,4,4_0,-1.1873862797394397,1.8061846196651459,1.1866742808371782,-3.9381285905838013,COMPLETED,Sobol
51,71.76,1.41232,5,5_0,-1.2908110505901278,-2.6293430477380753,1.5090201077051462,-3.9079870879650116,COMPLETED,Sobol
63,81.41,1.97048,6,6_0,-1.4372390598990024,-3.5276940017938614,0.13866510950028896,-0.2014627903699875,COMPLETED,Sobol
75,70.42,1.42805,7,7_0,-0.6009101951494813,1.9477206766605377,0.35404937993735075,-3.8243181481957436,COMPLETED,Sobol
87,81.23,1.58216,8,8_0,-1.1587896443300014,-3.147245186697784,2.0,-2.103060960812063,COMPLETED,MOO_Modular
99,81.05,1.56376,9,9_0,-0.3697828611708909,-4.0,2.0,-1.5277010322209188,COMPLETED,MOO_Modular
1,74.62,1.46788,10,10_0,-0.1910743091452185,-1.492823149190023,2.0,-1.8816301290055422,COMPLETED,MOO_Modular
2,81.5,1.97857,11,11_0,-0.10000000000000009,-4.0,2.0,0.6378100844139416,COMPLETED,MOO_Modular
4,74.71,1.44139,12,12_0,-0.444703358830455,-4.0,2.0,-3.0426408506761584,COMPLETED,MOO_Modular
5,85.61,1.74246,13,13_0,-0.9259970051391639,-3.5774007968355592,2.0,-1.1864592244705938,COMPLETED,MOO_Modular
6,76.5,1.49827,14,14_0,-0.39299945863485264,-3.43907696082381,1.3687793514370405,-2.29415229756314,COMPLETED,MOO_Modular
7,73.1,1.42698,15,15_0,-2.0,-0.48404433405659164,2.0,-3.1049726969230242,COMPLETED,MOO_Modular
8,73.46,1.43631,16,16_0,-1.0261509870946242,-1.0036306634676726,2.0,-2.9291719138883616,COMPLETED,MOO_Modular
9,73.01,1.44558,17,17_0,-0.10000000000000009,4.0,2.0,-1.6267918342033485,COMPLETED,MOO_Modular
10,69.44,1.39376,18,18_0,-2.0,4.0,2.0,-1.6950979964809019,COMPLETED,MOO_Modular
11,73.37,1.56619,19,19_0,-0.10000000000000009,4.0,0.1,-0.5810293871347891,COMPLETED,MOO_Modular
12,69.97,1.39718,20,20_0,-0.10000000000000009,4.0,2.0,-4.0,COMPLETED,MOO_Modular
13,71.4,1.40891,21,21_0,-2.0,-4.0,2.0,-3.8974262253932253,COMPLETED,MOO_Modular
14,78.28,1.50927,22,22_0,-0.40054394796896675,-3.848320282659581,2.0,-2.1091915477580003,COMPLETED,MOO_Modular
16,74.26,1.58903,23,23_0,-0.10000000000000009,-4.0,0.1,-4.0,COMPLETED,MOO_Modular
17,83.91,1.67678,24,24_0,-0.6771194576380519,-4.0,2.0,-1.382177629816208,COMPLETED,MOO_Modular
18,69.44,1.39376,25,25_0,-2.0,4.0,0.1,-4.0,COMPLETED,MOO_Modular
19,73.99,1.588,26,26_0,-0.10000000000000009,4.0,0.1,4.0,COMPLETED,MOO_Modular
20,70.24,1.39999,27,27_0,-0.10000000000000009,-4.0,2.0,-4.0,COMPLETED,MOO_Modular
21,76.68,1.75119,28,28_0,-0.10000000000000009,4.0,2.0,0.9737875171109627,COMPLETED,MOO_Modular
22,76.94,1.79591,29,29_0,-0.10000000000000009,4.0,2.0,4.0,COMPLETED,MOO_Modular
23,72.39,1.42522,30,30_0,-0.10000000000000009,1.6973464979534478,2.0,-2.4093313716373177,COMPLETED,MOO_Modular
24,69.88,1.39575,31,31_0,-0.825345864745924,3.4300134670620253,2.0,-2.385239757522452,COMPLETED,MOO_Modular
25,73.1,1.55822,32,32_0,-2.0,0.13092641909081326,0.1,-3.6620726404665285,COMPLETED,MOO_Modular
26,87.49,1.89561,33,33_0,-1.09596971051573,-3.258225682345572,2.0,-0.8608899376138144,COMPLETED,MOO_Modular
28,94.1,2.62271,34,34_0,-2.0,-2.807293920225037,2.0,0.23158555297026417,COMPLETED,MOO_Modular
29,90.88,2.23677,35,35_0,-1.535876905054716,-2.4529836663328233,1.7940885404357678,-0.16661310543450902,COMPLETED,MOO_Modular
30,98.3,4.68708,36,36_0,-2.0,-3.8855728099753377,1.9954423156228558,2.7804526878655604,COMPLETED,MOO_Modular
31,96.69,3.51037,37,37_0,-1.9876894772957283,-3.220069734749844,2.0,1.1933007289636466,COMPLETED,MOO_Modular
32,70.87,1.43566,38,38_0,-1.4872626477018964,1.30704336824419,0.8503638668814812,-0.41818869411146453,COMPLETED,MOO_Modular
33,75.34,1.55453,39,39_0,-0.9018374889961536,0.3836925882995006,2.0,-0.6426206704497499,COMPLETED,MOO_Modular
34,95.89,3.10531,40,40_0,-1.9989861749389823,-2.860143780866636,2.0,0.7411827037658645,COMPLETED,MOO_Modular
35,83.2,1.61886,41,41_0,-2.0,-2.5690376334223233,2.0,-1.899107561723273,COMPLETED,MOO_Modular
36,97.59,4.00286,42,42_0,-2.0,-2.738651538590582,2.0,1.8357606896807406,COMPLETED,MOO_Modular
37,75.87,1.63231,43,43_0,-1.2248379493014383,-0.5074536848796871,0.18542576593244192,-1.834775762007383,COMPLETED,MOO_Modular
38,69.44,1.39611,44,44_0,-1.0016004117782131,4.0,0.9409377580889559,-0.1077036119856718,COMPLETED,MOO_Modular
40,79.45,1.6413,45,45_0,-2.0,-0.9004271382150337,0.8517809420072795,-1.7086619734760085,COMPLETED,MOO_Modular
41,76.68,1.74264,46,46_0,-0.10000000000000009,-2.5554271243903828,0.1,4.0,COMPLETED,MOO_Modular
42,92.14,2.3838,47,47_0,-2.0,-2.7011087865406536,2.0,-0.10685349863267835,COMPLETED,MOO_Modular
43,71.85,1.50122,48,48_0,-0.10000000000000009,4.0,0.5094328408698012,-2.545737587852326,COMPLETED,MOO_Modular
44,71.76,1.52568,49,49_0,-0.42466635557821464,4.0,0.9416630581111926,2.584762677664295,COMPLETED,MOO_Modular
45,69.44,1.39385,50,50_0,-1.2800071224494345,4.0,2.0,-1.1713138336651605,COMPLETED,MOO_Modular
46,76.68,1.47218,51,51_0,-1.6996574996116816,-2.8353396439796708,2.0,-2.962359990323617,COMPLETED,MOO_Modular
47,69.44,1.39376,52,52_0,-1.5386612610812571,4.0,0.1,-0.8617905252798645,COMPLETED,MOO_Modular
48,69.97,1.40088,53,53_0,-0.45143214036099333,0.2937003171604813,1.5079062053130128,-4.0,COMPLETED,MOO_Modular
49,69.44,1.39376,54,54_0,-2.0,4.0,2.0,-4.0,COMPLETED,MOO_Modular
50,90.08,2.27104,55,55_0,-1.1256292285072664,-2.5713633485683673,1.8135232127915017,0.03869218014742515,COMPLETED,MOO_Modular
52,74.89,1.63799,56,56_0,-0.10000000000000009,0.2949364593714421,0.1,2.447032368274156,COMPLETED,MOO_Modular
53,76.05,1.49217,57,57_0,-1.3181095622851546,-0.3872895744636229,2.0,-1.6194477642617384,COMPLETED,MOO_Modular
54,80.43,1.96925,58,58_0,-0.10000000000000009,-1.4253367275763744,2.0,2.7986636436016292,COMPLETED,MOO_Modular
55,75.34,1.64252,59,59_0,-0.10000000000000009,-4.0,0.1,-1.4612947495047415,COMPLETED,MOO_Modular
56,78.37,1.81621,60,60_0,-2.0,-4.0,0.1,-4.0,COMPLETED,MOO_Modular
57,71.4,1.40695,61,61_0,-0.9186010254914647,-4.0,2.0,-3.9330995634575183,COMPLETED,MOO_Modular
58,70.6,1.40602,62,62_0,-1.657560319286517,-1.1504677345426586,2.0,-4.0,COMPLETED,MOO_Modular
59,94.46,2.78989,63,63_0,-1.9503269591378012,-3.053728362436439,2.0,0.43625822128423053,COMPLETED,MOO_Modular
60,97.86,4.21493,64,64_0,-2.0,-3.4677501622498212,1.967425585974302,2.0897909126185583,COMPLETED,MOO_Modular
61,71.31,1.50106,65,65_0,-1.0765664809241393,1.2748645785072057,0.1,0.7325135520469583,COMPLETED,MOO_Modular
62,69.44,1.39376,66,66_0,-2.0,4.0,0.97579165097729,-2.6641312268581347,COMPLETED,MOO_Modular
64,81.23,1.56496,67,67_0,-0.8210816629962903,-1.8764232023893963,2.0,-1.586492673115504,COMPLETED,MOO_Modular
65,73.73,1.44597,68,68_0,-0.10000000000000009,-4.0,1.7432328724999926,-2.484783988578388,COMPLETED,MOO_Modular
66,96.43,3.45126,69,69_0,-2.0,-2.8058589188262837,1.6346146894440277,1.4551760319414129,COMPLETED,MOO_Modular
67,69.44,1.39376,70,70_0,-1.2767706421124823,4.0,0.1,4.0,COMPLETED,MOO_Modular
68,73.1,1.55987,71,71_0,-0.710744229916457,1.1612903271666264,0.1,4.0,COMPLETED,MOO_Modular
69,73.01,1.51427,72,72_0,-0.10000000000000009,-0.9584753063203402,0.5916094587760374,-2.6139324072953025,COMPLETED,MOO_Modular
70,69.44,1.39376,73,73_0,-2.0,4.0,0.1,1.0928692994610314,COMPLETED,MOO_Modular
71,72.92,1.54477,74,74_0,-0.10000000000000009,4.0,0.1,-4.0,COMPLETED,MOO_Modular
72,69.53,1.39476,75,75_0,-1.109649134093959,4.0,0.1,-4.0,COMPLETED,MOO_Modular
73,69.44,1.39376,76,76_0,-2.0,4.0,0.8184430935801121,-0.6292563453315023,COMPLETED,MOO_Modular
74,76.59,1.54173,77,77_0,-0.10000000000000009,-2.375981401955187,2.0,-0.9759837826824556,COMPLETED,MOO_Modular
76,71.67,1.40987,78,78_0,-0.44531998032472764,-2.166915290769171,2.0,-3.5068558786215136,COMPLETED,MOO_Modular
77,71.76,1.41849,79,79_0,-0.5585929704084747,-4.0,1.1823192495405197,-3.8604602164961626,COMPLETED,MOO_Modular
78,89.81,2.1308,80,80_0,-1.4386397917114166,-2.4995435409371574,1.9934175190162875,-0.3682715383262325,COMPLETED,MOO_Modular
79,69.44,1.39611,81,81_0,-1.0487375340607463,4.0,0.1,2.577435754749046,COMPLETED,MOO_Modular
80,69.44,1.39376,82,82_0,-1.563967863926044,4.0,0.1,0.5219395304698784,COMPLETED,MOO_Modular
81,96.96,3.61686,83,83_0,-2.0,-2.8763376450566387,2.0,1.3792125019266246,COMPLETED,MOO_Modular
82,80.7,1.67353,84,84_0,-0.9702002089012445,-1.2054259901577842,1.1508084926651097,-0.9999511971189912,COMPLETED,MOO_Modular
83,94.19,2.809,85,85_0,-2.0,-2.528022209166185,1.8880524005828918,0.5281159894686418,COMPLETED,MOO_Modular
84,98.21,4.74251,86,86_0,-2.0,-3.141233687439919,1.9530929974246414,4.0,COMPLETED,MOO_Modular
85,69.53,1.3986,87,87_0,-0.7973134985328789,4.0,0.1,4.0,COMPLETED,MOO_Modular
86,97.77,4.314,88,88_0,-2.0,-2.9205943865874255,1.8886584363789163,2.5208282339028205,COMPLETED,MOO_Modular
88,80.16,1.96368,89,89_0,-2.0,-1.6469754019334988,0.1,-0.17219749821545038,COMPLETED,MOO_Modular
89,76.94,1.79591,90,90_0,-0.10000000000000009,4.0,2.0,2.537184782712327,COMPLETED,MOO_Modular
90,70.78,1.40272,91,91_0,-0.10000000000000009,2.2645835609493608,2.0,-3.398253742824194,COMPLETED,MOO_Modular
91,72.92,1.41672,92,92_0,-2.0,-2.2368232853389554,2.0,-3.5735463989404734,COMPLETED,MOO_Modular
92,69.44,1.39385,93,93_0,-0.915579739047486,4.0,2.0,-3.6181186326860044,COMPLETED,MOO_Modular
93,69.44,1.39376,94,94_0,-1.382410336731223,4.0,0.1,1.5305563225736973,COMPLETED,MOO_Modular
94,86.06,1.81912,95,95_0,-1.4130595098644994,-2.39783912116929,1.6733115038730138,-1.0589085305247,COMPLETED,MOO_Modular
95,69.44,1.39376,96,96_0,-1.505172185331624,4.0,0.9517288880097559,-0.6918224501193619,COMPLETED,MOO_Modular
96,70.15,1.40204,97,97_0,-2.0,1.0427815279646122,2.0,-2.072914438544724,COMPLETED,MOO_Modular
97,77.66,1.794,98,98_0,-0.8148243495482996,-2.754032994365001,0.1,-3.1759263893231915,COMPLETED,MOO_Modular
98,96.51,3.85663,99,99_0,-1.6646530312991825,-2.3159889984852464,1.87710744481482,2.5884732139179603,COMPLETED,MOO_Modular

Mar 03 '22 14:03 du-210

cc @sdaulton

Mar 03 '22 17:03 eytan

Hmm it kind of looks like Ax is minimizing both objectives in the later iterations.

What are the inferred objective thresholds? Do these seem reasonable.

What does the cross validation for each model look like? Is the model for one objective much more accurate than the model for the other? If one model is better than the other, then the candidates may be biased toward a region with more certainty about improvement.

How far off is this PF from the true PF? Mostly wondering if this is already nearly converged

Apr 08 '22 14:04 sdaulton

@du-210, do you have any thoughts on @sdaulton's comment above?

Apr 15 '22 20:04 lena-kashtelyan

Hi @sdaulton, sorry for the late reply.

The inferred thresholds seem reasonable. I have also tried setting them to fixed values [69, 5] but the optimal PF is still localised to a corner. However, by setting the thresholds to [85, 5], I get better diversity and convergence to the true PF.
Both objective models are quite accurate.
The PF converges to the true PF after ~30 iterations but it doesn't diversify over the entire true PF.

Apr 20 '22 07:04 du-210

Sorry for delay @du-210, @sdaulton do you have thoughts on how to proceed with figuring this out?

May 16 '22 22:05 lena-kashtelyan

Has this been resolved? My hunch is that that the PF is nearly converged and the model believes that the greatest potential improvements in hypervolume will come from the left bottom corner.

The coverage also looks pretty good along the identified PF. What does the true PF look like?

It is surprising that 85,5 works better than 69, 5, since the former would give no hypervolume to observed points with f0 <= 85. The former reference point could be useful for targeting the optimization towards the upper right corner in the later iterations.

What does the leave-one-out cross validation look like for each model? What is the noise level (inferred by the model or supplied to the model)?

A repro, if possible, would be very helpful. Thanks!

Sep 19 '22 22:09 sdaulton

cc @du-210

Sep 20 '22 15:09 lena-kashtelyan

Closing as inactive. Please reopen if following up.

Dec 06 '22 18:12 lena-kashtelyan