open-mmlab
Error in converting SlowFast model to onnx using tools/pytorch2onnx.py
Getting error like below while executing the conversion script:
python3 tools/pytorch2onnx.py /home/dev3/Documents/new_mmaction/mmaction2/work_dirs/ava/slowfast_kinetics_pretrained_r50_4x16x1_20e_ava_rgb_new-data_e20-2class/custom_slowfast.py /home/dev3/Documents/new_mmaction/mmaction2/work_dirs/ava/slowfast_kinetics_pretrained_r50_4x16x1_20e_ava_rgb_new-data_e20-2class/[email protected]_epoch_1.pth --is-localizer
/home/dev3/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/torch/nn/functional.py:3103: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details.
warnings.warn("The default behavior for interpolate/upsample with float scale_factor changed "
/home/dev3/mmdetection/mmdet/core/bbox/transforms.py:70: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if bboxes.size(0) > 0:
Traceback (most recent call last):
File "tools/pytorch2onnx.py", line 163, in
I don't think pytorch2onnx.py supports spatio-temporal action detection models.
@irvingzhang0512 thanks for quick response. if pytorch2onnx.py dosn't support then any other alternative to convert it onnx or tensorrt to optimize the model.
@irvingzhang0512 thanks for quick response. if pytorch2onnx.py dosn't support then any other alternative to convert it onnx or tensorrt to optimize the model.
Since spatio-temporal action detecction models use RoIAlign or RoIPool, it's not easy to convert pytorch models to onnx/tensorrt. You need build specific plugins to support RoIAlign/RoIPool(and other unsupported ops).
https://github.com/open-mmlab/mmcv/blob/48d990258549ca626fcf8c34488c00ed6fce108a/tests/test_ops/test_onnx.py#L150
@ZJU-lishuang Can you explain little bit about the link you gave above, How it is helpful in converting SlowFast spatio-temporal model to ONNX?
in mmcv,there is RoIAlign code for cuda,onnx and pytorch. As @irvingzhang0512 said,you can build specific plugins to support RoIAlign. And the link is the code about the RoIAlign. You can refer it to convert model to ONNX without write the code again.
@ZJU-lishuang sorry to bring you on the issue back, but i am confuse how to use RoIAlign function code provided in the link with pytorch2onnx.py file, to get it working for SlowFast temporal model.
@innerlee @ZJU-lishuang any input, how to use RoIAlign code with pytorch2onnx.py, for conversion.
For RoIAlign, I suggest consulting the mmdet team. They have some onnx experts.
@arvindchandel Hello. Have you solved this problem? Did you successfully switch slowfast to onnx？
I try to output slowfast to onnx too but seeing the same error. Would you please let me know if you figure it out. Thank you!
Hi @fengyaoluo @Crush-yq , No i could not find solution of it, plz post if you find some hack for it.
@arvindchandel Thank you for responding me! Our team ended up using Resnet18 trained on COCO to get the skeleton and then use LSTM trained on NTU dataset for action recognition as an alternative solution. In that sense, we are not using mmactions anymore but definitely hoping that they develop the feature to export to onnx in the future.
@fengyaoluo okk sure. I tried that earlier through skeletons, but as its 2 step process so causes performance issue.
Hi，I met the same problem, cannot export slowfast+R-CNN to onnx format. Did somebody figure it out ? I wonder whether it's ok to convert slowfast and R-CNN separately to onnx format. Thank you!
Im getting an error that is different from this one.. error logs posted in #1416 , Can anyone help?
Have found the solution? is it possible to convert them separately? They mentioned Slowfast on https://github.com/open-mmlab/mmaction2/blob/master/docs/tutorials/6_export_model.md as a supported model. How about using Yolo and SlowFast? YOLO can be found on ONNX.
@sakh251 but before SlowFast outputting detection results, it has to pass through roi_align and roi_pool layer to get detection result on multiple object. And since these 2 layers aren't supported by mmaction for conversion, I think we met the dead end here
