Sub-options in the config space

[tdeneau]

Apologies if this is answered elsewhere but assume:

• my config space has some general "Type" option, which say must be set to either A or B
• a bunch of boolean options A1, A2, A3, A4, ... An which only make sense if Type=A
• a bunch of boolean options B1, B2, B3, B4, ... Bn which only make sense if Type=B

I could of course have all the Ax and all the Bx options in the configuration space and then ignore the ones that do not apply to the chosen Type option. But I would assume that makes the manipulator search over a larger total space unnecessarily (especially if Type can have several different values). Or am I wrong in this assumption?

What is the best way to set up parameters to express this partitioning of the config space?

[phrb]

If the values of options when Type is A are completely independent of the values of options when Type is B, it seems that you have two separate search spaces.

Would it make sense in your case to just split the problem in two? Because from what you said it seems you want to find out the best configuration for Type A, and the best configuration for Type B, then pick the best configuration overall.

This would be equivalent to tuning just for A, then just for B, then picking the best one.

[tdeneau]

Yes, the Type A options don't even have any meaning when Type is B. An example for Type here could be a general Garbage Collection strategy in Java, each strategy having its own set of flags,

I see what you are saying about tuning for Type A and tuning for Type B separately, then picking the best. Of course there are many other options that likely don't depend on whether type is A or B, so for those options you'd be sort of searching the same space twice. But who knows maybe there is some small dependency of those other flags on the chosen Type.

[phrb]

Ok, so it seems it would really make more sense to tune each type separately, specially if you can't rule out interactions between type-independent and type-specific options.

An argument for doing that would be the following: Consider the search space when type is A. When you are in a good region of that space, the local search algorithms implemented in OpenTuner exploit this region using the assumption that good points are close to other good points. But if you change the type variable to B, in the case where it was a parameter of your configuration, you would end up in another region of a different search space, which could be really bad. You would then consider this new region as bad overall, even if it was a good region when type was A.

Additionally, you would have to find a way to fix all A parameters when type changes to B, because changing them when type is B would have no impact in your measured response and would only move you randomly in search space A, which means you could leave good regions of A.

This makes sense in your case?

[tdeneau]

Makes sense, I will give it a try.