pdq_evaluation Runitime

Hi,

for some reason evaluating a single image on ImageNet takes easily a whole minute. Is this normal? What should I expect?

Thanks!

Mar 24 '21 06:03 sigeisler

The amount of time PDQ takes per image is dependent largely on the number of detections you are producing as the most time intensive part is calculating the pixel-wise probability scores.

It is a fairly long process and evaluating over the entire PrOD Challenge Test dataset (https://competitions.codalab.org/competitions/20597) of about 56,000 images could often take 20 mins for competitors.

That said, 1 minute per image seems excessive.

Mar 26 '21 05:03 david2611

@sigeisler did you ever manage to figure out what was going on in your evaluation?

Apr 05 '21 23:04 david2611

Hi, thanks for your response. I will check it again by the end of next week I think and then will follow up. How many detections per image would you say is a reasonable number?

Apr 08 '21 09:04 sigeisler

to be getting 1 minute per image timings I would be (hopefully) expecting high hundreds of detections but I will be honest, I don't have a an accurate estimate for you. I haven't done timing tests in a while.

If you have anything below 100 detetions per image though I wouldn't be expecting it to take a minute per image.

Apr 08 '21 23:04 david2611