affordance-net icon indicating copy to clipboard operation
affordance-net copied to clipboard

Published 20 hours ago • verified publisher icon nqanh

way to calculate F score

Open fujenchu opened this issue 6 years ago • 6 comments

Hi Dr. Nguyen,

It is a really nice work! Thank you. Would you mind to provide script to compute the F score in the paper? I would like to evaluate the performance for my own dataset set. Or it is in the repo and I overlooked it? It would be great if I could borrow the code from you.

Thank you very much.

fujenchu avatar Jun 12 '18 15:06 fujenchu

Hi, please see https://github.com/nqanh/affordance-net/issues/19 and http://cgm.technion.ac.il/Computer-Graphics-Multimedia/Software/FGEval/

nqanh avatar Jun 13 '18 05:06 nqanh

Cool! Thanks for your reply. I just got it working.

fujenchu avatar Jun 13 '18 05:06 fujenchu

Hi Dr. Nguyen,

I just realized that previous works evaluated their approaches by first cropping images with ground truth mask for bounding boxes. I am wondering if you follow the same procedure (cropping the region of interest) or you pass the whole images for F score to get the results described in the paper? Thank you very much.

fujenchu avatar Aug 22 '18 21:08 fujenchu

We pass the whole image and evaluate F score. Please see paper for more details.

nqanh avatar Aug 23 '18 07:08 nqanh

I see. Thank you very much! I've read the paper and we are wondering how many epochs you have trained on UMD dataset for getting the results? It seems the hyperparameters in the repo are for IIT. Would you mind to provide the hyperparameters for UMD if possible? It seems stepsize of 150k in solver.prototxt is large for UMD given the dataset size. Thank you in advance!

fujenchu avatar Aug 23 '18 19:08 fujenchu

You can just train then stop when the lost doesn't decrease anymore :)

nqanh avatar Aug 23 '18 20:08 nqanh