ActionDetection-AFSD icon indicating copy to clipboard operation
ActionDetection-AFSD copied to clipboard

thumos14_gt.json

Open lurchycc opened this issue 2 years ago • 8 comments

I noticed that the number of videos in the thumos14_gt. json file is 410, it seems that there are 3 videos missing in the test part, now I check that 'video_test_0000270' is not in the thumos14_gt. json, does this affect the evaluation result?

lurchycc avatar Apr 10 '22 03:04 lurchycc

@lurchycc good issue! In experiments, we directly took the THUMOS GT file from GTAD codes (https://github.com/frostinassiky/gtad/blob/master/evaluation/thumos_gt.json). To compare to GTAD, we did not change the json file. You can update the thumos14_gt.json and check the new results.

linchuming avatar Apr 13 '22 12:04 linchuming

Okay, thanks for your reply.

lurchycc avatar Apr 13 '22 12:04 lurchycc

@lurchycc good issue! In experiments, we directly took the THUMOS GT file from GTAD codes (https://github.com/frostinassiky/gtad/blob/master/evaluation/thumos_gt.json). To compare to GTAD, we did not change the json file. You can update the thumos14_gt.json and check the new results.

the question that I asked: Hello, do you remove the background data provided by the thumos 14 dataset during training and testing? I mean in the thumos14 annotation files :val_Annotation_ours.csv、test_Annotation_ours.csv and thumos_gt.json, you remove the background data:Ambigous class. I see that GTAD don‘t remove them.

menghuaa avatar Apr 27 '22 11:04 menghuaa

@menghuaa, I think you means that we remove the 'Ambiguous' class label. These data label is useless to training and testing for action detection task. GTAD is an action proposal task which predicts the foreground proposal and do not need the class label, so they can use the data sample with 'Ambiguous' class. In detection evaluation, GTAD also removes the 'Ambiguous' class data label according to their thumos_gt.json.

linchuming avatar Apr 27 '22 11:04 linchuming

@menghuaa,我认为您的意思是我们删除了“模棱两可”的类别标签。这些数据标签对于动作检测任务的训练和测试是无用的。GTAD 是一个动作提议任务,它预测前景提议并且不需要类标签,因此他们可以使用具有“模糊”类的数据样本。在检测评估中,GTAD 还根据他们的 thumos_gt.json 删除了“模糊”类数据标签。

thanks for your reply,for action proposal task,can I also remove the the data sample with 'Ambiguous' class in traning and testing?

menghuaa avatar Apr 27 '22 11:04 menghuaa

@menghuaa you can observe the gt file for ambiguous class to action proposal task. If ambiguous proposals are included in gt file, you should not remove them in training and testing.

linchuming avatar Apr 27 '22 13:04 linchuming

@menghuaa you can observe the gt file for ambiguous class to action proposal task. If ambiguous proposals are included in gt file, you should not remove them in training and testing.

ok,thanks for you reply. As an additional question, have you ever seen the features of thumos14 fine tuned on the thumos14 dataset?

menghuaa avatar Apr 27 '22 13:04 menghuaa

@menghuaa no, I have never seen it.

linchuming avatar Apr 27 '22 13:04 linchuming