How evaluate the performance on MegaFace?

[lymanblue]

Hi~

Thank you for your great work!

The megaface devkit requires 2 files (i.e., megaface and facescrub) for performance evaluation. However, the eval_megaface.py generates .feat in each folder. (do we wrongly used the script?) How do we evaluate the performance on megaface?

Thank you.

[wujiyang]

The feature format is generated following the test protocol of megaface, you can use megaface devkit tool to evaluate your model.

[lymanblue]

The megaface devkit requires only 2 feature files for evaluation.

But we get .feat file for each image separately in each folder. Do we use eval_megaface.py incorrectly?

[wujiyang]

Megaface devkit requires two folders not two files. Did you use the official devkit tools?

[lymanblue]

I see. I think that we misunderstand the usage of the official devkit.

Just for clarification. Is it normal to scatter the .feat files with the corresponding images?

Thank you.

[wujiyang]

Yes，the devkit tool will find corresponding image features base on the image list json file

[lymanblue]

So if we want to submit our result to the MegaFace website. Could we just simply compress all the image+.feat files and upload the zipped files to the website?

[wujiyang]

Just uploading the featues only is ok.

[lymanblue]

Thank you very much.

Just one additional question: We get only 3.6k images after applying align_facescrub.py to FaceScrub dataset. Is this a correct result? (The python script is adopted from InsightFace.)

Again, thank you for your kindly responses!

[wujiyang]

I suggest that you should totally understand the test protocol of MegaFace first before the following operations.

In MegaFace Challenge 1, for the probe set facescrub, only 3530 images of 80 individial identities are required, and for gallery set, there exist different splits of distractors, which corresponding to 10, 100, 1000, 10000, 100000, 1000000.

All the images needed in this test are listed in megaface_features_list.json

[lymanblue]

We have read the readme of MegaFace, while we are still not sure for some details. Therefore we just double confirmed the details to make sure the process is correct.

Thank you for your great help!