result of post processing

[fyangneil]

i use the provided model and post processing code on davis16 dataset, the final result is 69.3 which is lower than 71.5 reported in paper. I am not sure if the result is correct.

[antonilo]

I think you might have an issue with image sizes. Are you sure you are running the post-processing on the final resolution (640x480)? Did you use our provided script to compute the metric or did you implement your own?

[fyangneil]

@antonilo I tried again and still got 69.3 on DAVIS The output mask is 480x854 not 640x480. I did use your provided model and post-processing code and did not change anything.

[antonilo]

Do you get the same results as in here http://rpg.ifi.uzh.ch/data/detection_results.zip ? Did you use this evaluation script? If not, I might need more details to debug the problem (TensorFlow version used, for example)

[fyangneil]

I did use your provided evaluation script. but I do not think I get the exactly same detection results as yours, because based on your detection results and evaluation tool, I can get the number reported in paper.

---

From: Antonio Loquercio [email protected] Sent: Monday, October 26, 2020 12:37 PM To: antonilo/unsupervised_detection [email protected] Cc: Fan Yang [email protected]; Author [email protected] Subject: [External] Re: [antonilo/unsupervised_detection] result of post processing (#11)

Do you get the same results as in here http://rpg.ifi.uzh.ch/data/detection_results.zip ? Did you use this evaluation scripthttps://github.com/antonilo/unsupervised_detection/blob/master/test_generator.py? If not, I might need more details to debug the problem (TensorFlow version used, for example)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/antonilo/unsupervised_detection/issues/11#issuecomment-716671831, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALAUWTU27SKZGKU6RCI33GTSMWQULANCNFSM4SMHJSJA.

[antonilo]

Might be that you used a sub-optimal checkpoint. How many epochs did you train?

[fyangneil]

I use the model you provided in github

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Antonio Loquercio [email protected] Sent: Tuesday, October 27, 2020 4:55:08 AM To: antonilo/unsupervised_detection [email protected] Cc: Fan Yang [email protected]; Author [email protected] Subject: [External] Re: [antonilo/unsupervised_detection] result of post processing (#11)

Might be that you used a sub-optimal checkpoint. How many epochs did you train?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/antonilo/unsupervised_detection/issues/11#issuecomment-716817269, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALAUWTSWATX3SPWDXJVNHPLSMXO2ZANCNFSM4SMHJSJA.

[WeidiXie]

@fyangneil Hi, anything follows this ? have you reproduced the same number ? eg. 71 ?

[fyangneil]

no, I cannot reproduce the number reported in paper

---

From: Weidi [email protected] Sent: Friday, February 5, 2021 5:49 PM To: antonilo/unsupervised_detection [email protected] Cc: Fan Yang [email protected]; Mention [email protected] Subject: [External] Re: [antonilo/unsupervised_detection] result of post processing (#11)

@fyangneilhttps://github.com/fyangneil Hi, anything follows this ? have you reproduced the same number ? eg. 71 ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/antonilo/unsupervised_detection/issues/11#issuecomment-774330070, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALAUWTR44TO3FIZCSZEZRYDS5RYX3ANCNFSM4SMHJSJA.

[antonilo]

Sorry, but it is very difficult to help without details. I would say that the problem is in the way images are pre-processed (reshaping and central cutting). The changes might have affected the ckpts. What if you train a new ckpt (from scratch or resuming from the one available)? The predicted masks have been produced with the model available online.

[fyangneil]

But, I did not change anything. I simply used the model online and run the test script for DAVIS dataset, the result is lower than the number reported in paper.

---

From: Antonio Loquercio [email protected] Sent: Saturday, February 6, 2021 1:43 PM To: antonilo/unsupervised_detection [email protected] Cc: Fan Yang [email protected]; Mention [email protected] Subject: [External] Re: [antonilo/unsupervised_detection] result of post processing (#11)

Sorry, but it is very difficult to help without details. I would say that the problem is in the way images are pre-processed (reshaping and central cutting). The changes might have affected the ckpts. What if you train a new ckpt (from scratch or resuming from the one available)? The predicted masks have been produced with the model available online.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/antonilo/unsupervised_detection/issues/11#issuecomment-774522838, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALAUWTSVMTQ64UBN5E7MKB3S5WEVTANCNFSM4SMHJSJA.

[fyangneil]

@WeidiXie Have you reproduced the number reported in paper?

[fyangneil]

Do you get the same results as in here http://rpg.ifi.uzh.ch/data/detection_results.zip ? Did you use this evaluation script? If not, I might need more details to debug the problem (TensorFlow version used, for example) I used the script. the tensorflow version is 13.1.

[WeidiXie]

I see, thanks, I was just wondering the step-wise breakdown results, but I'll test this myself when I got time.