Face_Pytorch
Face_Pytorch copied to clipboard
wujiyang
Questions about the performance of megaface with @R notation
Hi~
Is the symbol @R represent the cleaned megaface dataset as stated in insighface paper? The performance is improved significantly with symbol R in your experiments.
We tried the cleaned script, but it only cleaned 24 files which the improvement is little. We would like to know the effect of the cleaned megaface if symbol R represents the noise removal.
Thank you.
We tried, but it only cleaned 24 files and the improvement is little. The overlapped files is so few that we want to know this overlapped amount is correct or not.
24 is only the number of noisy files in facescrub.
You should also do feature refinement for megface files.
Our result is based on the feature refinement of megaface and facescrub at the same time.
We trained MobileFaceNet on CAISA dataset (i.e., small protocol). The MF Acc. is around 68% before removing noises. (similar to the result reported in this repo). After removing noises, the MF Acc. remains around 68%.
On the other hand, maybe we just misunderstand the values of MF Acc. and Ver. Is MF Acc. the value in roc section of cmc file corresponding to around 1e-6? If not. Where we can find these metrics (MF Acc. and Ver.)?
Thank you for your great help!
Yes, you are right about the MF Acc. and MF Ver.
Acc. means rank 1 identification rate and Ver. means TPR@FPR=1e-6.
Maybe you need this: facescrub_noises.txt and megaface_noises.txt
The result is already based the suggested noise list.
According to your comment, I find that our original interpretation of MF Acc. may be wrong. Is MF Acc the printed Rank-1 accuracy after running run_experiment? (i.e., "Rank 1: 0.63", then the MF Acc is 63%). And Ver. is TPR@FPR=1e-6 listed in cmc file, which is 68% is our case.
While the behavior is not similar to the reported results. The MF Acc. remains in the same level even after noise removal.
We will continue to find out the reasons. Thank you very much.