pytorch2onnx.torch2onnx failed after the nms message

[yeungkong]

Hi developers. I encounted error when I try to convert my model to tensorrt. My environment is Win10, Cuda11.3, VS2017, cudnn8.2.0, Python3.7, Pytorch1.11.0, mmcv-full1.4.8, mmdet2.23.0.

I use the command.

python ./tools/deploy.py \
configs/mmdet/detection/detection_tensorrt_dynamic-64x64-608x608.py \
../mmdetection/configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco2.py \
../mmdetection/work_dirs_maskrcnn/epoch_340.pth \
../mmdetection/demo/tensorrt_demo.png\
--work-dir ../mmdeploy/work_dir \
--show  \
--device cuda:0 \
--log-level INFO

The following shows the error message.

2022-08-25 17:05:10,324 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
load checkpoint from local path: ..mmdetection/work_dirs_maskrcnn/epoch_340.pth
..\mmdetection\mmdet\datasets\utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  'data pipeline in your config file.', UserWarning)
2022-08-25 17:05:16,489 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
2022-08-25 17:05:16,490 - mmdeploy - INFO - Export PyTorch model to ONNX: ..mmdeploy/work_dir\end2end.onnx.
2022-08-25 17:05:16,530 - mmdeploy - WARNING - Can not find torch._C._jit_pass_onnx_deduplicate_initializers, function rewrite will not be applied
..\mmdeploy\mmdeploy\core\optimizers\function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treat
ed as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
..\mmdetection\mmdet\models\backbones\cbam.py:76: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This res
ults in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  x = x.view(batch, 2, rchannel // 2, height, width)
..\mmdetection\mmdet\models\dense_heads\rpn_head.py:108: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). T
his results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  rpn_cls_score = rpn_cls_score.view(batch, rchannel // 2, 2, *rpn_cls_score.shape[2:])
..\mmdetection\mmdet\models\dense_heads\anchor_head.py:123: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead
  warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
..\mmdetection\mmdet\core\anchor\anchor_generator.py:333: UserWarning: ``grid_anchors`` would be deprecated soon. Please use ``grid_priors``
  warnings.warn('``grid_anchors`` would be deprecated soon. '
..\mmdetection\mmdet\core\anchor\anchor_generator.py:370: UserWarning: ``single_level_grid_anchors`` would be deprecated soon. Please use ``single_level_grid_priors``
  '``single_level_grid_anchors`` would be deprecated soon. '
..\mmdeploy\mmdeploy\codebase\mmdet\models\dense_heads\rpn_head.py:78: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value wil
l be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
..\mmdeploy\mmdeploy\pytorch\functions\topk.py:56: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a co
nstant in the future. This means that the trace might not generalize to other inputs!
  if k > size:
..\mmdeploy\mmdeploy\codebase\mmdet\core\bbox\delta_xywh_bbox_coder.py:39: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value
 will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(0) == bboxes.size(0)
..\mmdeploy\mmdeploy\codebase\mmdet\core\bbox\delta_xywh_bbox_coder.py:41: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value
 will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(1) == bboxes.size(1)
..\mmdeploy\mmdeploy\codebase\mmdet\deploy\utils.py:47: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead
 to errors or silently give incorrect results.
  assert len(max_shape) == 2, '`max_shape` should be [h, w]'
..\mmdeploy\mmdeploy\codebase\mmdet\core\post_processing\bbox_nms.py:259: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value
will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
..\mmdeploy\mmdeploy\mmcv\ops\nms.py:178: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in
 the future. This means that the trace might not generalize to other inputs!
  out_boxes = min(num_boxes, after_topk)
..\mmdeploy\mmdeploy\mmcv\ops\nms.py:181: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in
 the future. This means that the trace might not generalize to other inputs!
  (batch_size, out_boxes)).to(scores.device))
2022-08-25 17:05:22,364 - mmdeploy - ERROR - `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed. exit.

How can I fix the problem? Thank you in advance.

[irexyc]

Hi @yeungkong , have you installed the onnxruntime package? If not, you can install it through pip install onnxruntime and try again.

[yeungkong]

I did not install onnxruntime before. I installed it and tried it again and it started to run. Thank you.

[yeungkong]

@irexyc I am confused as I am using the tensorrt backend. Shouldn't I need not install the onnxruntime package? It does not convert the model successfully in the process afterwards.

[irexyc]

We have no clue to the onnxruntime problem for now.

You should use configs in configs/mmdet/instance-seg/

[yeungkong]

@irexyc I think I have finally converted the model successfully. Although I do not know what is the problem, hope you can find out something from what I did(#953).