How to align the predicted label with ground truth?

[TiankaiHang]

Hi, Thanks for your amazing work!! I just wonder how to align the predicted label with the ground truth. For example, given an input and fixed network, we could do the prediction (fixed) . We could use the argmax function as pseudo labels during training period. However, how could we calculate the mIoU metric with ground truth and predicted results. as we could assign the pixel belonging to a dog to label "0" when annotating the data set, we could also assign it to label "1", it's not sure.

Thanks~

[kanezaki]

Hi,

Please see #2 for our evaluation code.

[kanezaki]

Yup, the iou with a blank image will be rather high when the ground truth segments are large. Here are the comparison of mIOU with a blank image and our method.

BSD500:             Blank    Proposed All        0.0458    0.3050 Fine      0.0249    0.2592 Coarse  0.1001    0.3739

VOC2012: Blank    Proposed    Proposed+scribbles 0.2972    0.3520        0.6174

Thanks, Asako

On 2021/01/05 9:03, Tiankai Hang wrote:

As what you have done, if the output result is just a black image, the iou will be very high. So i don't think your evaluation code is reasonable.🙊

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kanezaki/pytorch-unsupervised-segmentation-tip/issues/6#issuecomment-754299366, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFU3GJSZ3ZQMC55D4MHV4LSYJJO5ANCNFSM4VNFQAOQ.

[TiankaiHang]

Oh, thanks for your kind answer!! But why the blank result is so low (near zero, supposed to be near 1).... (Maybe my question is a little bit stupid).

[kanezaki]

A blank image can be regarded as a single large segment. IoU of a large segment and a small segment is low (near zero).