Sync Algorithm Settings names for various Optimisation Frameworks

[andreyvelich]

/kind feature

Since Katib supports various frameworks with the same algorithms: https://www.kubeflow.org/docs/components/katib/katib-config/#suggestion-settings, we should try to be consistent with Algorithm Settings names.

For example, n_EI_candidates in TPE algorithm in Hyperopt, and n_ei_candidates in TPE algorithm in Optuna. We have to validate all supported algorithms.

cc @gaocegege @johnugeorge @c-bata @g-votte

[g-votte]

Sounds good. Actually, I was wondering which name I should adopt when creating #1613 .

I summarized below the correspondence of the algorithm setting among the Goptuna, Optuna, and Hyperopt services. We may want to fix the inconsistency especially around TPE.

TPE Parameters

• We can align ei_candidates and n_EI_candidates.
• gamma and prior_weight have been supported only with the Hyperopt service so far. We can think about exposing those parameters for Goptuna and Optuna.
• The behavior around startup_trials is different. With Optuna and Goptuna, the parameter is supported as an argument of the algorithm setting. In the Hyperopt service, the value is fixed to request_number instead.

goptuna	optuna	hyperopt
random_state	random_state	random_state
startup_trials	startup_trials	not supported (request_number is passed to hyperopt instead)
ei_candidates	ei_candidates	n_EI_candidates
not supported	not supported	gamma
not supported	not supported	prior_weight

CMA-ES Parameters

• The parameter names are consistent b/w Goptuna and Optuna.
• CMA-ES is not supported with the Hyperopt service.

goptuna	optuna	hyperopt
random_state	random_state	N/A
sigma	sigma	N/A
restart_strategy	restart_strategy	N/A

Random Search Parameters

• The parameter name is consistent among Goptuna, Optuna, and Hyperopt.

goptuna	optuna	hyperopt
random_state	random_state	random_state

[andreyvelich]

Thank you for summarising this @g-votte! I will take a look closely after the Katib 0.12 release on that.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[andreyvelich]

/lifecycle frozen