issue-tracking icon indicating copy to clipboard operation
issue-tracking copied to clipboard

`API.get_metrics_summary(metric=name)` returns an empty list instead of dict

Open neighthan opened this issue 4 years ago • 1 comments

If you call API.get_metrics_summary with the name of a particular metric, and the experiment has data for that metric, a dictionary is returned. If the experiment lacks data for that metric, then an empty list is returned. An empty dictionary would make more sense here so that the return type in this case doesn't depend on whether the metric exists or not. This makes further processing easier because you don't have to handle two different possible return types. As an example, here's what I would like to do:

from comet_ml.api import API, Parameter
api = API()
workspace = "..."
project = "..."
query = Parameter("something") == "another thing"

exps = api.query(workspace, project, query)
metric_name = "return"
default = float("nan")
max_metrics = [exp.get_metrics_summary(metric_name).get("valueMax", default) for exp in exps]

but if the metric might not exist on some experiments, then I have to handle empty lists instead by doing something like

max_metrics = []
for exp in exps:
    metric = exp.get_metrics_summary(metric_name)
    if isinstance(metric, list):
        max_metrics.append(default)
    else:
        max_metrics.append(metric["valueMax"])

neighthan avatar Jun 10 '20 01:06 neighthan

Yes, good idea!

dsblank avatar Jun 10 '20 01:06 dsblank

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Nov 08 '23 21:11 github-actions[bot]

This issue was closed because it has been stalled for 5 days with no activity.

github-actions[bot] avatar Nov 14 '23 21:11 github-actions[bot]