Adding two dependent parameters

[daothanhtuan]

The tutorial shows how to add independent parameters clearly. But I couldn't find how to add dependent (integer) parameters. For example, if I want to add X and Y but there is a constraint X + Y < 100.

How should I do this?

Thank you

[jansel]

You can do this by sub-classing IntegerParameter and overriding the legal_range method. For example:

class DependentIntegerParameter(IntegerParameter):
    def legal_range(self, config):
        if self.name == 'a':
            max_value = 99 - config['b']
        else:
            assert self.name == 'b'
            max_value = 99 - config['a']
        min_value = 0
        return min_value, max_value

Here is that a full working example.

Note that it is not always a great idea to have dependent parameters as it creates a more difficult search space. If you can come up with a projection of your search space that removes dependencies those projections can be easier to autotune. For example if you have the dependency a+b=100. It is better to have a single parameter a then derive b=100-a. You could also apply your constraints after the fact by having float parameters with unit ranges in the config then mapping those to the legal ranges.

[daothanhtuan]

Thank you for your information. I would possibly try all your options to see what fits the best. Could you please elaborate the later cases (removing the constraint and using float parameters) a little bit more?

[jansel]

Hrmm...

An alternate version would be something like this. Note that I changed your example from a + b < 100 to a + b + c = 100 with is equivalent for non-negative a, b, and positive c.

The key code there is replacing a dynamic legal_range with a normalize method:

class DependentIntegerParameter(IntegerParameter):
    def normalize(self, config):
        total = config['a'] + config['b'] + config['c']
        config['a'] = 100 * config['a'] // total
        config['b'] = 100 * config['b'] // total
        config['c'] = 100 * config['c'] // total

This change allows more freedom in the search space, allowing temporary illegal configurations which are then fixed in normalization.

This representation would have an advantage in that it would be easier to transition from, for example, a=99, b=0 to a=90, b=9. In the prior representation, if the search technique first looked at b it would see a legal range of [0,0] and would be unable to make any changes to it. In the first representation this change would require first decreasing a (possibly a locally sub-optimal choice) then increasing b. This new representation allows this transition to happen in a single step. Increasing b will automatically scale a down in normalization.