affordance-net icon indicating copy to clipboard operation
affordance-net copied to clipboard

how to measure the performance.

Open ambl2357 opened this issue 7 years ago • 15 comments

I do not know how to measure the performance. Could you tell me a file or a method?

ambl2357 avatar Feb 22 '18 05:02 ambl2357

You just need to merge all the outputted affordance maps into one map (please see the paper for details). Then use the Weighted F-measure code from here to evaluate the results with the groundtruths.

nqanh avatar Feb 22 '18 08:02 nqanh

Clearly resolved. thank you and have a good day

-----Original Message----- From: "Anh Nguyen"[email protected] To: "nqanh/affordance-net"[email protected]; Cc: "ambl2357"[email protected]; "Author"[email protected]; Sent: 2018-02-22 (목) 17:42:31 Subject: Re: [nqanh/affordance-net] how to measure the performance. (#18)

You just need to merge all the outputted affordance maps into one map (please see the paper for details). Then use the Weighted F-measure code from here to evaluate the results with the groundtruths. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

ambl2357 avatar Feb 22 '18 08:02 ambl2357

HI, what does mean by"merge all the outputted affordance maps into one map"? How to calculate one kind of affordance like 'grasp'? There are more than one affordances on one pic? How to calculate F measure separately?

ClaireTun avatar May 01 '18 09:05 ClaireTun

The network outputs each affordance map for each detected object (as in the demo). Since the image may has many objects --> we need to merge them into "one predicted" image. Then the F measure will calculate the accuracy for each affordance (on this "merged" output).

nqanh avatar May 02 '18 00:05 nqanh

HI, Nqanh~ about the Fmeasure for each affordance, I still have questions. In the Fmeasure code , the type of GT is logical, which means the background and the foreground. But in foreground, we have different objects and affordances. Do I have to seperate all the affordances and make new GTs?

ClaireTun avatar May 08 '18 01:05 ClaireTun

We can save all the affordances in 1 map. Then, for each affordance, we select its id and the background --> it becomes logical when compares with the GT.

nqanh avatar May 08 '18 23:05 nqanh

Well,ok! Thank you for your answer and patience! But I am still a little confused. Would you plz share this part code,like how to save in 1 map and select id?

ClaireTun avatar May 10 '18 00:05 ClaireTun

Please see https://github.com/nqanh/affordance-net/issues/19 (also please use the search function, most of the problems we have already be answered. Thanks!)

nqanh avatar May 10 '18 00:05 nqanh

Wow~ Thank you!!

ClaireTun avatar May 10 '18 01:05 ClaireTun

The network outputs each affordance map for each detected object (as in the demo). Since the image may has many objects --> we need to merge them into "one predicted" image. Then the F measure will calculate the accuracy for each affordance (on this "merged" output).

Hi nqanh, Please help me by teaching me how to "merge affordance map" into one map. Could you tell me more details about that? For example, we have 3 file .pnp (contains affordance maps), and 3 file .sm, so which extension file (sm or pnp) we will use to merge and how to merge it, please?

chaundm avatar Aug 21 '19 13:08 chaundm

im sorry, I studied so long ago and now I'm doing something different so I can't answer the question. One thing I could answer was that there was no phenomenon corresponding to the question. -----Original Message----- From: "chaundm"[email protected] To: "nqanh/affordance-net"[email protected]; Cc: "jay-sik"[email protected]; "Author"[email protected]; Sent: 2019-08-21 (수) 22:18:36 (GMT+09:00) Subject: Re: [nqanh/affordance-net] how to measure the performance. (#18)

The network outputs each affordance map for each detected object (as in the demo). Since the image may has many objects --> we need to merge them into "one predicted" image. Then the F measure will calculate the accuracy for each affordance (on this "merged" output).

Hi nqanh, Please help me by teaching me how to "merge affordance map" into one map. Could you tell me more details about that? For example, we have 3 file .pnp (contains affordance maps), and 3 file .sm, so which extension file (sm or pnp) we will use to merge and how to merge it, please? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

ambl2357 avatar Aug 21 '19 13:08 ambl2357

Wow~ Thank you!!

Hi ClaireTun, I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please?

chaundm avatar Aug 30 '19 06:08 chaundm

Well, I finished the project half a year ago. Some details,  I have already forgot. Next week,I will return to school .I can review the code and try to answer your questions 

------------------ Original ------------------ From: chaundm <[email protected]> Date: Fri,Aug 30,2019 2:54 PM To: nqanh/affordance-net <[email protected]> Cc: ClaireTun <[email protected]>, Comment <[email protected]> Subject: Re: [nqanh/affordance-net] how to measure the performance. (#18)

Wow~ Thank you!!

Hi ClaireTun, I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

ClaireTun avatar Aug 30 '19 14:08 ClaireTun

Wow~ Thank you!!

Hi ClaireTun, I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please? Hi chaundm, did you have solved the problem?

Rechardgu avatar Jan 17 '20 04:01 Rechardgu

Well, I finished the project half a year ago. Some details,  I have already forgot. Next week,I will return to school .I can review the code and try to answer your questions  ------------------ Original ------------------ From: chaundm <[email protected]> Date: Fri,Aug 30,2019 2:54 PM To: nqanh/affordance-net <[email protected]> Cc: ClaireTun <[email protected]>, Comment <[email protected]> Subject: Re: [nqanh/affordance-net] how to measure the performance. (#18) Wow~ Thank you!! Hi ClaireTun, I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Hi ClaireTun, my ground truth and predicted map are png files with each masks with affordance color. How to use the weighted F_measure?

Rechardgu avatar Jan 17 '20 04:01 Rechardgu