How to obtain AUC (test) results

Open try-harder12 opened this issue 1 year ago • 1 comments

Hello, I encountered some issues while reproducing the results of your work on the Activity_net dataset. How can I obtain the AUC (test) results? I can get AUC (val) by running the command python main.py --cfg-file config/anet_proposals_CLIP_v1.yaml MODE 'validation' GPU_IDS [0], but I cannot get AUC (test) by running python main.py --cfg-file config/anet_proposals_CLIP_v1.yaml MODE 'testing' GPU_IDS [0]. I don't know how to do it. Can you tell me?

Jul 23 '24 02:07 try-harder12

Hi,

As annotations on test sub-set is not made publicly available by the organizer of ActivityNet-1.3, we could not do evaluation on testing set. Instead, we use our model to do inference on testing videos and upload the results to organizer's website for evaluation.

I'm concerned that the submission portal is closed for a while (may be I'm wrong). However, I have seen many of recent works only reported their performances on validation set of ActivityNet-1.3 due to difficulties in obtaining scores for testing set.

Jul 23 '24 04:07 vhvkhoa