MTMT icon indicating copy to clipboard operation
MTMT copied to clipboard

Some error in your shadow detection results at test datasets

Open Jingwei-Liao opened this issue 4 years ago • 4 comments

hi bro,we think you may upload wrong result on SUB, because we find the SUB result is the same as the SBU_crf result

Jingwei-Liao avatar Oct 20 '20 13:10 Jingwei-Liao

And can you upload your pretrained shadow detection model, we need it to compare with my method on our own dataset

Jingwei-Liao avatar Oct 20 '20 13:10 Jingwei-Liao

Q1: We generate the outputs again and get the similar results. The reason of the comparability of SBU and SBU_crf maybe lead by that we use the binarization operation (prediction = (prediction>90)*255) before CRF in SBU dataset. Because at that time, we observed that many pixels are positive but still under 127.5, we adjust it by binarization operation with bias. However, after the submission, we find that we can use the weighted BCE loss to make the balance this problem. Q2: Thanks for you advise, We will upload our pretrained model soon.

eraserNut avatar Oct 20 '20 15:10 eraserNut

Q1: We generate the outputs again and get the similar results. The reason of the comparability of SBU and SBU_crf maybe lead by that we use the binarization operation (prediction = (prediction>90)*255) before CRF in SBU dataset. Because at that time, we observed that many pixels are positive but still under 127.5, we adjust it by binarization operation with bias. However, after the submission, we find that we can use the weighted BCE loss to make the balance this problem. Q2: Thanks for you advise, We will upload our pretrained model soon.

hello, I wonder that whether you select some different thresholds on different dataset. For example, may you select (prediction>90) in SBU, and (prediction>x) in ISTD(x!=90)? Thanks!

guanhuankang avatar Oct 22 '20 08:10 guanhuankang

Actually, we just do this binary operation in SBU. For UCF and ISTD, we save the soft output before crf.

eraserNut avatar Oct 23 '20 06:10 eraserNut