Re-computing f-scores reported in paper

[alexrich021]

Hi there,

Thanks for the great work!

I'm trying to re-compute the metrics reported in Table 4 of your paper (prec., rec., f-score on YFCC evaluation) but there doesn't appear to be code in your repo for doing this. I have my own function for computing the metrics, but am a bit confused about setting a threshold.

First, do you have the thresholds you used for each YFCC scene stored anywhere? I tried using the values stored in the text files in yfcc_data/gt_resolution, but using your provided trained models with this threshold doesn't give me the same results you report in Table 4.

Second, is Eq. 7 in the paper correct? I understand the goal with Eq. 7, but shouldn't the distance argument be ||K^-1 (D(p)p) - K^-1 (D(p')p')||. This way, you find the median distance in scene space between back-projected points 2 pixels away from each other. Is this perhaps what you used or did you use Eq. 7 as is?

Thanks! Alex