tsn-pytorch

tsn-pytorch copied to clipboard

Would you share the trained models in pytorch?

Good point. I will give the models on UCF101 split1 for your references.

Thanks! I‘m very expect for it!

@yjxiong Hi, thank you for your wonderful job!!! Could you tell me where to download the pretrained models in pytorch???

@yjxiong Hi, Xiong. Also need the pretrained models in pytorch. Thanks so much!

We don't have all pretrained UCF101/HMDB51 models in the PyTorch format for download. I can convert the Caffe pretraned models in the original TSN codebase to PyTorch format in next few weeks.

I have finished a complicated script for transferring caffemodel to pytorch. When I use the transferred pytorchmodel on tsn-pytorch code, I have got the same result as paper. If you desire pytorchmodel immediately, please see ‘’https://github.com/gss-ucas/caffe2pytorch-tsn'‘

@gss-ucas Thanks

@yjxiong Thanks for your efforts. I recently parsed your pretrained caffemodel (ucf split 1) into tensorflow with google protobuf. And I constructed the rgb stream in tensorflow with every layer identical to the one in your caffe train-val protocal. However, the accuracy is still 10% lower than the caffe version. (Padding strategy is different in caffe and tf, but it is properly solved by manually padding in tf.) I'm wondering is there any other details I should take care? Thanks!

BTW, @gss-ucas it seems that maxpooling with floor mode is used in your implementation, which is not consistent with the caffe version. But strange you can still reproduce the results lol

@scenarios what data augmentations are you using? Is your model BN-Inception (+TSN)? If so, are you enabling partial BN when finetuning?

Victor Hugo

On Fri, Nov 3, 2017 at 6:07 AM, scenarios [email protected] wrote:

@yjxiong https://github.com/yjxiong Thanks for your efforts. I recently parsed your pretrained caffemodel (ucf split 1) into tensorflow with google protobuf. And I constructed the rgb stream in tensorflow with every layer identical to the one in your caffe train-val protocal. However, the accuracy is still 10% lower than the caffe version. (Padding strategy is different in caffe and tf, but it is properly solved by manually padding in tf.) I'm wondering is there any other details I should take care? Thanks!

BTW, @gss-ucas https://github.com/gss-ucas it seems that maxpooling with floor mode is used in your implementation, which is not consistent with the caffe version. But strange you can still reproduce the results lol

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/yjxiong/tsn-pytorch/issues/6#issuecomment-341641372, or mute the thread https://github.com/notifications/unsubscribe-auth/ADoEbq73ODXXGhJgSLkhn7smgpjf6Vr6ks5sysm6gaJpZM4PCiVp .

@victorhcm Actually I'm not finetuning. I simply initialize the my BN-Inception model in tensorflow with the parameter released by TSN (it is a caffemodel, I parsed it using protobuf) and do online test without any training. At the very begining, the accuracy is only around 0.6 with 3 segment. Then I realized there is a slight difference between caffe and tensorflow in padding (both for convolution and pooling). After modifying the pading , the accuracy increased to 71%, which is still 10% lower than the caffe resuts.(I test the same model with TSN home-brewed caffe and got 0.81 accuracy with 3 segment). I have double-checked every layer and for sure they are identical to the model definition in train_val protocal in TSN (or there must be errors when loading the parameter). Still confused why..

@victorhcm And for testing, I resize the frame to 256x340 and central crop to 224x224. Mean is subtracted. And I also change rgb image to bgr format for consistence.

@yjxiong I use the default command you provided to train my RGBDiff、RGB and Flow model with tsn-pytorch, could you please tell me if they were initialized on ImageNet?

@JiqiangZhou Yes.

@yjxiong thank you! I will read the code carefully.

I run the default command and obtain a lower performance than the paper(85.12 vs 85.7) on the rgb stream. The model is available at http://pan.baidu.com/s/1eSvo8BS . Hope it helps.

@yjxiong Even I trained the models using default settings for split-1 of the UCF-101 dataset and am getting a lower performance than reported. Below are the numbers I got:

RGB - 85.57% Flow - 84.26% RGB + Flow - 90.44%

The main difference seems to be because of the Flow stream, what could be the reason for this?

@utsavgarg Please follow the instructions in Caffe TSN to extract optical flow.

https://github.com/yjxiong/tsn-pytorch/issues/30

@yjxiong I had done that, I extracted optical flows using the extract_optical_flow.sh script included with Caffe TSN but I used extract_cpu instead of extract_gpu in build_of.py, will that cause this difference in performance?

Yes that's the problem. Always use extract_gpu. The optical flow algorithm you need is not available in extract_cpu.

@utsavgarg extract_gpu use TVL1 optical flow and it leads to a better accuracy. I did a experiment for that.

Okay thanks.

Did anyone get the pytorch pretrained models on ActivityNet or Kinetics? @yjxiong @nationalflag @gss-ucas @ntuyt @scenarios

Hello, are there any available models trained in ufc?

Thanks!

@Ivorra please see https://github.com/gss-ucas/caffe2pytorch-tsn

@gss-ucas Thanks for kindly answering, I am having troubles in "test_models.py", as after loading your model "ucf101_rgb.pth" it seems that some keys do not exist, for with the command example:

python test_models.py ucf101 RGB <ucf101_rgb_val_list> ucf101_rgb.pth --arch BNInception --save_scores <score_file_name>

I get the error:

/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py:514: UserWarning: src is not broadcastable to dst, but they have the same number of elements. Falling back to deprecated pointwise behavior. own_state[name].copy_(param) Traceback (most recent call last): File "test_models.py", line 54, in print("model epoch {} best prec@1: {}".format(checkpoint['epoch'], checkpoint['best_prec1'])) KeyError: 'epoch'

And if I comment that line, another key error appears:

/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py:514: UserWarning: src is not broadcastable to dst, but they have the same number of elements. Falling back to deprecated pointwise behavior. own_state[name].copy_(param) Traceback (most recent call last): File "test_models.py", line 56, in base_dict = {'.'.join(k.split('.')[1:]): v for k,v in list(checkpoint['state_dict'].items())} KeyError: 'state_dict'

Thanks!

I wonder how to fuse? who can help me？ thanks

@gss-ucas Alrights, I did not noticed the test_models.py file in your repository. I just substituted it buy the one this repository has. Moreover, accuracy for UCF101 - Fold 1 works as it should, given your provided converted models and the reference at the official project page:

• RGB accuracy: 86.013%
• Flow accuracy: 87.698%
• Final accuracy: 93.798%

Thanks!

@poppingcode I followed the instructions at the original TSN repository. Just copied the eval_scores.py file provided there in conjunction with the folder pyActionRecog which is needed as a dependency.

Hope it works fine for you!

@gss-ucas Hi, did you manage to convert kinetics pretrained models (as shown at http://yjxiong.me/others/kinetics_action/ ) to pytorch too?

Maybe this Caffe --> PyTorch model convertor is worth looking at:

https://github.com/marvis/pytorch-caffe