video_features icon indicating copy to clipboard operation
video_features copied to clipboard

Could you tell me the kinetics's details in accuracy of you project

Open Violetgyw opened this issue 3 years ago • 5 comments

Violetgyw avatar May 11 '21 09:05 Violetgyw

I haven't evaluated the pre-trained models on Kinetics. The feature extraction relies on the original pre-trained checkpoints. Therefore, the quality should be the same as of the original models.

In case of R(2+1)D RGB-only, it should match the reported one in here. One thing, however, it would be difficult to obtain the same quality of features for I3D because in the original paper, the model uses TVL1 algorithm for optical flow frame calculation. In this repo, there are two faster options to calculate the optical flow: PWC and RAFT.

I would be nice if someone could evaluate the quality of the methods from this repo by training the classification layer, esp I3D with different flow extractors.

v-iashin avatar May 11 '21 10:05 v-iashin

By the way, another option on how to evaluate the performance would be to use --show_pred argument from here which shows the predictions on the kinetics (classes and probs). But one still needs to change the script and download the dataset

v-iashin avatar May 11 '21 10:05 v-iashin

By the way, another option on how to evaluate the performance would be to use --show_pred argument from here which shows the predictions on the kinetics (classes and probs). But one still needs to change the script and download the dataset

Thank you for your answer, I tried to use TV-L1, RAFT, PWC, the sample v_CricketShot_g04_c01.mp4 provided at https://github.com/deepmind/kinetics-i3d, using the same processing method, I found only TV-L1 Which category can be accurately distinguished, the official i3d uses TV-L1, other optical flow distributions are different, the effect is not good as it should be

Violetgyw avatar May 12 '21 06:05 Violetgyw

Can you provide a code snippet? I am afraid, I have a hard time understanding what you mean.

v-iashin avatar May 12 '21 07:05 v-iashin

@Violetgyw Please share which repo you used for I3D with TV-L1?

varungupta31 avatar Aug 19 '23 07:08 varungupta31