kubeflow
Whether the hyperparameter search algorithm will refer to the value of additionalMetricNames
If not, is the purpose of the additionalMetricNames parameter just for visualization? It will not affect the experimental results？ Please help me answer this question, thank you very much !!!🙏
That is correct @YKL2436542696
Currently, additionalMetricNames is only used for metrics tracking purposes. So you can get them in the Katib UI or using Katib SDK. We have an open issue to support multi-objective optimization, but it has not been implemented yet: https://github.com/kubeflow/katib/issues/1549
@andreyvelich Thank you。 Can I understand it this way? The strategies for additionalMetricNames in Spec.Objective.MetricStrategies currently have no practical effect. (This may work when multi-objective optimization is supported in the future)
And I noticed that the metrics values output by the "/katib/fetch_hp_job_info/" interface in the katib-ui module are based on spec.Objective.Type and have nothing to do with Spec.Objective.MetricStrategies。 I'm curious why the metric values are not output according to the strategy defined by Spec.Objective.MetricStrategies 🤯
The strategies for additionalMetricNames in Spec.Objective.MetricStrategies currently have no practical effect.
That's right, we check if Experiment goal is reached based on objective metric name and its metrics strategy: https://github.com/kubeflow/katib/blob/master/pkg/controller.v1beta1/experiment/util/status_util.go#L166
However, we send those metrics to the suggestion service based on the strategies: https://github.com/kubeflow/katib/blob/master/pkg/controller.v1beta1/suggestion/suggestionclient/suggestionclient.go#L419. So if user implements custom algorithm service that analyses all Trial metrics that might give some value.
I'm curious why the metric values are not output according to the strategy defined by Spec.Objective.MetricStrategies
It's a good point, actually UI should just take Trial metrics results from .status.observation and based on strategies should take the appropriate value, not from metrics logs as we do right now: https://github.com/kubeflow/katib/blob/master/pkg/ui/v1beta1/hp.go#L156-L166
Like what we do for Suggestion Service.
Thanks for reporting!
/bug /area frontend /good-first-issue
@andreyvelich: This request has been marked as suitable for new contributors.
Please ensure the request meets the requirements listed here.
If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue command.
In response to this:
I'm curious why the metric values are not output according to the strategy defined by Spec.Objective.MetricStrategies
It's a good point, actually UI should show just take Trial metrics results from
.status.observation, not from metrics logs as we do right now: https://github.com/kubeflow/katib/blob/master/pkg/ui/v1beta1/hp.go#L156-L166 Thanks for reporting!/bug /area frontend /good-first-issue
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
![google-oss-prow[bot] avatar](https://avatars.githubusercontent.com/in/143327?v=4 "Published by a pub.dev verified publisher"){kind=small}
Thank you for your interest @xr-dev-saurabh, as I can see @live2awesome is already working on this issue. Please feel free to work on other issues that don't have assignee /assign @live2awesome