multi_object_datasets icon indicating copy to clipboard operation
multi_object_datasets copied to clipboard

Segmentation metrics is nan when n_true_groups == 1

Open wetliu opened this issue 5 years ago • 0 comments

Hello. According to the comment:

# The case where n_true_groups == n_pred_groups == 1 needs to be
   # special-cased (to return 1) as the above formula gives a divide-by-zero.
# This might not work when true_mask has values that do not sum to one:

I understand both_single_cluster is handling this corner case.

What I am confused is this function: https://github.com/deepmind/multi_object_datasets/blob/master/segmentation_metrics.py#L97. It seems to me it makes sense when it is applied to ground truth masks as _all_equal(true_group_ids) , but when it is applied to _all_equal(pred_group_ids), the prediction slot for object may not necessarily be "the first slot" as suggested by values[..., :1].

May I ask if it is corner case missing in this implementation?

Thank you!

More specifically, your code first finds the argmax of the tensor with shape [bs, n_pixels, n_classes] over the last axis. Assume n_classes == 3, then because my prediction for object is in slot 3, I believe that argmax will return two values, 3 for the pixels with objects and 1 for pixels exclude the object, as argmax returns 1 by default if all values are equal. It seems to me that this will lead to a wrong condition in your ARI value.

wetliu avatar Sep 19 '19 19:09 wetliu