Legal values for hyperparameter

[janvanrijn]

Constants can not be a boolean, neither can they be None. Is there any reason for this? I think there are many cases where we want them to be either of these.

Traceback (most recent call last):
  File "/home/janvanrijn/projects/sklearn-bot/examples/run_on_task.py", line 39, in <module>
    run()
  File "/home/janvanrijn/projects/sklearn-bot/examples/run_on_task.py", line 32, in run
    configuration_space = sklearnbot.config_spaces.get_config_space(args.classifier, None)
  File "/home/janvanrijn/projects/sklearn-bot/sklearnbot/config_spaces/bootstrap.py", line 19, in get_config_space
    return sklearnbot.config_spaces.decision_tree.get_hyperparameter_search_space(seed)
  File "/home/janvanrijn/projects/sklearn-bot/sklearnbot/config_spaces/decision_tree.py", line 21, in get_hyperparameter_search_space
    'max_leaf_nodes', None, meta={'component': 'decisiontreeclassifier'})
  File "ConfigSpace/hyperparameters.pyx", line 122, in ConfigSpace.hyperparameters.Constant.__init__
TypeError: Constant value is of type <class 'NoneType'>, but only the following types are allowed: (<class 'int'>, <class 'float'>, <class 'str'>)

[janvanrijn]

If you agree, I can contribute with a PR

[mfeurer]

The current reason is that the ConfigSpace only handled the pcs format in the beginning, which does not allow something to be None or bool. If you want to add support for that, I'd be happy to accept a PR, but keep in mind that you'd have to touch cython code and should add type checks to the two pcs exporters.

[janvanrijn]

I would probably be happy to add this (no commitment on a time line yet)

Is the PSC format documented somewhere? Is it something that is defined in Freiburg, or commonly used across research labs?

[janvanrijn]

Interestingly, boolean values (True, False) seem to be OK for the CategoricalHyperparameter. This covers most of my use-cases.

[mfeurer]

Is the PSC format documented somewhere?

Yes: http://www.cs.ubc.ca/labs/beta/Projects/SMAC/

Is it something that is defined in Freiburg, or commonly used across research labs?

There's a version defined by ACLIB which is used by other labs which do algorithm configuration, too.

Interestingly, boolean values (True, False) seem to be OK for the CategoricalHyperparameter

Interesting, I'll keep this open so we can add a unit test at some ponit in time.