BDCN icon indicating copy to clipboard operation
BDCN copied to clipboard

Cannot reproduce the reported F-score

Open ForawardStar opened this issue 3 years ago • 9 comments

Can anyone successfully reproduce the results reported in this paper?My python environment is python 3.6, and train the model using the augmented dataset downloaded from http://vcl.ucsd.edu/hed/HED-BSDS.tar. But the visual results are totally not edges. I solely change the data path with the rest unchanged.

ForawardStar avatar May 29 '21 08:05 ForawardStar

Can anyone successfully reproduce the results reported in this paper?My python environment is python 3.6, and train the model using the augmented dataset downloaded from http://vcl.ucsd.edu/hed/HED-BSDS.tar. But the visual results are totally not edges. I solely change the data path with the rest unchanged.

ForawardStar avatar Jun 01 '21 12:06 ForawardStar

I got it!! The default setting of this code is to randomly initialize VGG16, Once I provide the pre-trained VGG16, the output edges is reasonable. But, the ODS-F is still lower than the reported value in the paper, which is 0.798. I wonder if there's anything else I need to pay attention to?

ForawardStar avatar Jun 01 '21 12:06 ForawardStar

Thanks for your attention of our work. We have shown the training details in implement detail which using data augmentation same as HED, and the hyperparamter yita is 0.3. Moreover, model is pretrained on imagenet, while lr , weight decay, optimizer, batchsize and lr schedulaer all are listed in the implement detail section.

pkuCactus avatar Jun 04 '21 15:06 pkuCactus

@pkuCactus What about the training dataset? Are your model trained with augmentated PASCAL VOC, how to obtain such data?

ForawardStar avatar Jun 07 '21 06:06 ForawardStar

作者您好,我得到的结果也没有论文中给的高,用的是您提供的预处理模型,在仅用baseline训练的时候得到的模型测试结果最高才0.71,而您论文中给的是0.76左右。关于GT处理有个疑问,是先将几个标注的做平均得到概率图,然后设置yita=0.3, 这句代码[gt >= self.yita] = 1是对的么?

wenya1994 avatar Nov 15 '21 12:11 wenya1994

@ForawardStar We used the Pascal VOC Context(also called Pascal Context) for augmentation. It can be obtained from the official website https://cs.stanford.edu/~roozbeh/pascal-context/

pkuCactus avatar Nov 15 '21 13:11 pkuCactus

@young532 我想你说的是关于消融实验部分的结果我们在文章中也说了,是在BSDS500的训练集上训练,在BSDS500的验证集上的验证结果。关于GT就是先将几个标注的做平均得到概率图,然后按照代码中提供的处理方式处理的。

pkuCactus avatar Nov 15 '21 13:11 pkuCactus

Hello, can we have some communication @young532? I also want to run this code, but there are some problems. My email is: [email protected] , hope to get your help.

mitldyjiushijie avatar Jun 08 '22 09:06 mitldyjiushijie

I got it!! The default setting of this code is to randomly initialize VGG16, Once I provide the pre-trained VGG16, the output edges is reasonable. But, the ODS-F is still lower than the reported value in the paper, which is 0.798. I wonder if there's anything else I need to pay attention to?

hello,i the ODS-F i reproduced is 0.784,its lower than the reported,i wonder if some codes have been changed,i only change the data path and yita.

liuxingyu123 avatar Sep 23 '22 11:09 liuxingyu123