Google Colab: MishCudaFunction() problems when exporting ONNX

[blah12629]

I have successfully trained a model on a Google Colab runtime, but when I try to export it, I get an error about MishCudaFunction().

This is what I have done:

• pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 -f https://download.pytorch.org/whl/torch_stable.html
• pip install git+https://github.com/JunnYu/mish-cuda.git
• pip install -U onnx
• Download trained weights.
• Copy models/export.py to the ScaledYOLOv4 root.
• Run export.py

Exporting failed with this error message: ONNX export failure: ONNX export failed: Couldn't export Python operator MishCudaFunction

I tired using pip install git+https://github.com/thomasbrandon/mish-cuda instead of JunnYu's mish-cuda but the error is still there.

I have also tried this code from https://github.com/linghu8812/tensorrt_inference/issues/13#issuecomment-747821470 but it still gives the same error message.

Here is a colab link for reproduction: https://colab.research.google.com/drive/1_2h6_Swb62DqLGOWgt8d8EyOj3wtyqlE

[WongKinYiu]

https://github.com/WongKinYiu/ScaledYOLOv4/issues/56

[blah12629]

Thank you for the response, but the issue has still not been solved. I have also raised an issue on the referred repository.

Here are the changes I have made: https://colab.research.google.com/drive/1-Xn99ZMBHTj1h5Yo8Xjz9TiWbFwLEP41

[WongKinYiu]

following the steps, i can export.

1. get yolov4-large

!git clone https://github.com/linghu8812/ScaledYOLOv4.git

%cd ScaledYOLOv4/
!git fetch origin yolov4-large:yolov4-large
!git checkout yolov4-large

2. modify the code in models/common.py

#from mc.build.lib.mish_mish import MishCuda as Mish
from mish_cuda import MishCuda as Mish

3. install mish_cuda

!git clone https://github.com/JunnYu/mish-cuda
%cd mish-cuda
!python setup.py build install
%cd ..

4. prepare requirement and files

!pip install onnx-simplifier
!curl -c ./cookie -s -L "https://drive.google.com/uc?export=download&id=1aXZZE999sHMP1gev60XhNChtHPRMH3Fz" > /dev/null
!curl -Lb ./cookie "https://drive.google.com/uc?export=download&confirm=`awk '/download/ {print $NF}' ./cookie`&id=1aXZZE999sHMP1gev60XhNChtHPRMH3Fz" -o yolov4-p5.pt
!rm ./cookie
from shutil import copyfile
copyfile('models/export_onnx.py',  'export_onnx.py')

5. export model

!python3 export_onnx.py --weights yolov4-p5.pt

[EconML]

@WongKinYiu why is the mish cuda git repo different here than it is in the tutorial to run the model. i.e JunnyYu vs thomasbrandon? Thx

[WongKinYiu]

JunnyYu's repo support higher torch/cuda/cudnn version.

[anhtuanlinux]

Thank you for the response, but the issue has still not been solved. I have also raised an issue on the referred repository.

Here are the changes I have made: https://colab.research.google.com/drive/1-Xn99ZMBHTj1h5Yo8Xjz9TiWbFwLEP41

Hi @blah12629, With Colab, I has the same Mish Cuda lib problem before with https://github.com/JunnYu/mish-cuda as well as https://github.com/thomasbrandon/mish-cuda.git. I found that, when I run git and install command step by step like:

!git clone https://github.com/JunnYu/mish-cuda %cd mish-cuda !python setup.py build install %cd ..

mish_cuda would be installed in path: /usr/local/lib/python3.7/dist-packages/mish_cuda-0.0.3-py3.7-linux-x86_64.egg/mish_cuda. Therefore, we can't import this lib by "from mish_cuda import ...." Replacing, I run the following command: !pip install git+https://github.com/JunnYu/mish-cuda.git, mish_cuda is installed at '/usr/local/lib/python3.7/dist-packages/mish_cuda'

I tried with https://github.com/thomasbrandon/mish-cuda.git It also works similarly when you import "from mish_cuda import...".

I think this problem comes from the way "install" command extract file to the lib folder in local.

[sakulh]

Thank you @WongKinYiu . I used you command and everything is OK. I generated *.onnx file. But after convertion to tensorrt than I see this shape in output:

platform: "tensorrt_plan"
max_batch_size: 1
input {
  name: "images"
  data_type: TYPE_FP32
  dims: 3
  dims: 896
  dims: 896
}
output {
  name: "output"
  data_type: TYPE_FP32
  dims: 65856
  dims: 85
}
default_model_filename: "model.plan"

Is this output shape [1, 65856, 85] correct? Thank you

[romil611]

Hi, @WongKinYiu I followed the exact steps. I'm still not able to successfully export it to onnx. @sakulh did you require any changes? I just cloned the scaled yolo repo, installed mish-cuda(by both the original method as well as the one anhtuanlinux suggested), downloaded the weights and then ran the export.py present in ./models/ I get the following:

Starting ONNX export with onnx 1.7.0... Fusing layers... Model Summary: 417 layers, 1.2753e+08 parameters, 1.23263e+08 gradients ONNX export failure: ONNX export failed: Couldn't export Python operator MishCudaFunction

Defined at:
/home/dreamvu/anaconda3/lib/python3.8/site-packages/mish_cuda-0.0.3-py3.8-linux-x86_64.egg/mish_cuda/__init__.py(20): forward
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(704): _slow_forward
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(720): _call_impl
/home/dreamvu/romil/yolo/ScaledYOLOv4/models/common.py(34): fuseforward
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(704): _slow_forward
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(720): _call_impl
/home/dreamvu/romil/yolo/ScaledYOLOv4/models/yolo.py(129): forward_once
/home/dreamvu/romil/yolo/ScaledYOLOv4/models/yolo.py(109): forward
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(704): _slow_forward
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(720): _call_impl
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/jit/__init__.py(412): wrapper
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/jit/__init__.py(421): forward
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(722): _call_impl
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/jit/__init__.py(338): _get_trace_graph
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/onnx/utils.py(319): _trace_and_get_graph_from_model
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/onnx/utils.py(366): _model_to_graph
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/onnx/utils.py(526): _export
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/onnx/utils.py(86): export
/home/dreamvu/anaconda3/lib/python3.8/site-packages/torch/onnx/__init__.py(203): export
export.py(44): <module>


Graph we tried to export:
graph(%images : Float(1:4915200, 3:1638400, 1280:1280, 1280:1),
      %model.0.conv.weight : Float(32:27, 3:9, 3:3, 3:1),
      %model.0.conv.bias : Float(32:1),
      .......