The definition of config_constraints and metric_constraints are not clear in Docs

[YIWEI-CHEN]

There are two constraints can be used in tuning procedure,config_constraints and metric_constraints. What is the definition of config_constraints? When we should use config_constraints? What difference between config_constraints and metric_constraints? To my understanding, config_constraint is used to restrict the specific attribute of a trial's config (e.g., area). The attribute of the config is constant during the trial training. On the other hand, metric_constraints is used to constrain the performance of a trial (e.g., precision). The metric values fluctuate as the training. They are not constant. After I check the Doc, it seems not to explain their difference explicitly. Please enhance the documents. Thank you.

[qingyun-wu]

Thank you @YIWEI-CHEN for raising this issue. They are indeed not clearly explained in the current documentation.

Definitions:

• config_constraints is aList[Tuple[Callable[[dict], float], str, float]], meaning that the constraints are functions that needs to be provided by the user, e.g., the area function.

• While the metric_constraints is a List[Tuple[str, str, float]]. The actual calculation of the constraints should be embedded in the metric function and be reported accordingly. The user needs to specify the constraints name (in string format) in metric_constraints.

The differences and when should you use which:

• The key difference is that the calculation of constraints in config_constraints does not rely on the evaluation procedure in the metric function. For example, when a constraint only depends on the config itself.
• Another difference is that constraints inconfig_constraints will be checked before evaluation, so if a config does not satisfy config_constraintsit will not be evaluated.

Let me know if they are still unclear to you. And can you please also help improve the documentation accordingly? Thank you!