katib
katib copied to clipboard
kubeflow
Deprecate Skopt Suggestion Service
/kind feature
Describe the solution you'd like [A clear and concise description of what you want to happen.] Unfortunately, the scikit-optimize was closed on February 29, 2024.
https://github.com/scikit-optimize/scikit-optimize
So we need to stop supporting the skopt suggestion service as well. I think that we can follow similar deprecation steps as the MXNet.
Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.]
Love this feature? Give it a 👍 We prioritize the features with the most 👍
/unassign @PeterWrighten
We need to follow the deprecation flow step by step.
/unassign @PeterWrighten
We need to follow the deprecation flow step by step.
Got it. Thanks!
That is sad to hear @tenzen-y.
Do we have any alternative to use Bayesian Optimization in Hyperopt or Optuna ?
That is sad to hear @tenzen-y.
Do we have any alternative to use Bayesian Optimization in Hyperopt or Optuna ?
Unfortunately, both optuna and Hyperopt don't provide Bayesian sampler.
- https://optuna.readthedocs.io/en/stable/reference/samplers/index.html
- https://hyperopt.github.io/hyperopt/#algorithms
Maybe we could check some Open Source BO implementations. For example: https://github.com/bayesian-optimization/BayesianOptimization
Also, quite recently @contramundum53 added GPSampler support in Optuna: https://github.com/optuna/optuna/pull/5185
Maybe we can re-use it. cc @c-bata @a9p for your ideas here.
Also, quite recently @contramundum53 added GPSampler support in Optuna: https://github.com/optuna/optuna/pull/5185
Oh, I didn't find that.
@andreyvelich @tenzen-y I'm glad that you are considering integration of Optuna's GPSampler into katib!
GPSampler is planned to be released in Optuna v3.6, our next minor release. It is still experimental and future breaking changes/bugs are possible (the code is only a month old!), but if it's okay for you, we'll be happy to hear that.
@andreyvelich @tenzen-y I'm glad that you are considering integration of Optuna's
GPSamplerinto katib!GPSampleris planned to be released in Optuna v3.6, our next minor release. It is still experimental and future breaking changes/bugs are possible (the code is only a month old!), but if it's okay for you, we'll be happy to hear that.
Oh, I see. Thank you for letting us know @contramundum53
Is there a graduation plan for GPSampler? For example, did the optuna community decide when GPSampler graduates to stable?
Sorry no, that will depend on many factors, such as how much the new feature is used, how many feature requests come, how many bugs we encounter, and so on.
Thanks for letting us know @contramundum53!
@tenzen-y @johnugeorge What do you think about it ? Should we use skopt for 1 Katib/Kubeflow release and add deprecation notice for users who will run experiments with bayesianoptimization algorithm ?
Also, implement GPSampler in Optuna Katib Suggestion and explain users how to update Katib Config to use BO from Optuna rather than from skopt ?
After some time we will remove skopt and fully rely on Optuna for bayesianoptimization algorithm ?
Sorry no, that will depend on many factors, such as how much the new feature is used, how many feature requests come, how many bugs we encounter, and so on.
@contramundum53 Thank you for letting me know!
@tenzen-y @johnugeorge What do you think about it ? Should we use skopt for 1 Katib/Kubeflow release and add deprecation notice for users who will run experiments with bayesianoptimization algorithm ? Also, implement GPSampler in Optuna Katib Suggestion and explain users how to update Katib Config to use BO from Optuna rather than from skopt ? After some time we will remove skopt and fully rely on Optuna for bayesianoptimization algorithm ?
@andreyvelich I totally agree with you.
@tenzen-y Should we try to add deprecation notice before the next release ?
@tenzen-y Should we try to add deprecation notice before the next release ?
@andreyvelich Shouldn't be marked as release 0.17, not 0.16?