Internal Error when trying to export torch onnx model to tensorrt

[PhaniShekhar]

Description

I am trying to convert a onnx model created from a torchvision.maskrcnn_resnet50_fpn to tensorrt using the command: "trtexec --onnx=model.onnx --saveEngine=model.trt"

The conversion fails with the following error:

[10/21/2022-00:04:53] [E] Error[4]: [shuffleNode.cpp::symbolicExecute::392] Error Code 4: Internal Error (Reshape_82: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2]) [10/21/2022-00:04:53] [E] [TRT] ModelImporter.cpp:773: While parsing node number 93 [Pad -> "onnx::Unsqueeze_413"]: [10/21/2022-00:04:53] [E] [TRT] ModelImporter.cpp:774: --- Begin node --- [10/21/2022-00:04:53] [E] [TRT] ModelImporter.cpp:775: input: "image" input: "onnx::Pad_411" input: "onnx::Pad_412" output: "onnx::Unsqueeze_413" name: "Pad_93" op_type: "Pad" attribute { name: "mode" s: "constant" type: STRING }

[10/21/2022-00:04:53] [E] [TRT] ModelImporter.cpp:776: --- End node --- [10/21/2022-00:04:53] [E] [TRT] ModelImporter.cpp:779: ERROR: ModelImporter.cpp:180 In function parseGraph: [6] Invalid Node - Pad_93 [shuffleNode.cpp::symbolicExecute::392] Error Code 4: Internal Error (Reshape_82: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2]) [10/21/2022-00:04:53] [E] Failed to parse onnx file [10/21/2022-00:04:53] [I] Finish parsing network model [10/21/2022-00:04:53] [E] Parsing model failed [10/21/2022-00:04:53] [E] Failed to create engine from model or file. [10/21/2022-00:04:53] [E] Engine set up failed

Environment

TensorRT Version: 8.4.3 ONNX-TensorRT Version / Branch: main GPU Type: Tesla K80 Nvidia Driver Version: 470.141.03 CUDA Version: 11.4 CUDNN Version: 8.2.1 Operating System + Version: Ubuntu 20.04.4 Python Version (if applicable): 3.8.5 TensorFlow + TF2ONNX Version (if applicable): N/A PyTorch Version (if applicable): 1.11.0 Baremetal or Container (if container which image + tag):

Relevant Files

Steps To Reproduce

[bmfire1]

do u have any sulotion? same question with u.

[zhenhuaw-me]

Hi @PhaniShekhar Could you please help to share the link to the ONNX model?

[isaacraf95]

any updates on this issue?

[zhenhuaw-me]

@isaacraf95 @bmfire1 @PhaniShekhar Could you please share your ONNX model? You may narrow down to a minimal reproducible model if you want. Thanks!

[isaacraf95]

of course! this is the link, thanks in advance! could you share your process?

[zhenhuaw-me]

@isaacraf95 It seems your model violates the constraint of TensorRT. Are you seeing erros like this?

[03/14/2023-06:34:47] [E] [TRT] ModelImporter.cpp:776: ERROR: ModelImporter.cpp:195 In function parseGraph:
[6] Invalid Node - /roi_heads/box_roi_pool/If
/roi_heads/box_roi_pool/If_OutputLayer: IIfConditionalOutputLayer inputs must have the same shape. Shapes are [-1] and [-1,1].

Taking the If node /roi_heads/box_roi_pool/If as example, if you check the subgraphs (jit3 and jit4) of the branches, you will see that the shapes of the output of them are different.

You need to fix this in your Torch code.

[zhenhuaw-me]

@PhaniShekhar If this is still an issue for you?

[isaacraf95]

@zhenhuaw-me that is not my error, when I try with the latest version of tensorRT(8.5.3.1), I get the following error:

[6] Invalid Node - /roi_heads/Reshape [graphShapeAnalyzer.cpp::analyzeShapes::1872] Error Code 4: Miscellaneous (IShuffleLayer /roi_heads/Reshape: reshape wildcard -1 has infinite number of solutions or no solution. Reshaping [0,12] to [0,-1].)

name: "/roi_heads/Reshape" op_type: "Reshape" attribute { name: "allowzero" i: 0 type: INT }

do you know how to deal with this? or some advice? thanks in advance!

[zhenhuaw-me]

@isaacraf95 8.5 gives me different error. I suggest to create another issue for MarkRCNN since the "internal error" is not specific enough. And please share the environment and the command you use - you will see a template when creating new "Bug report" issue: https://github.com/onnx/onnx-tensorrt/issues/new?assignees=&labels=&template=bug_report.md&title=

Thanks!

[03/16/2023-02:36:53] [E] [TRT] ModelImporter.cpp:732: ERROR: ModelImporter.cpp:172 In function parseGraph:
[6] Invalid Node - /rpn/anchor_generator/ConstantOfShape
[network.cpp::setWeightsName::3366] Error Code 1: Internal Error (Error: Weights of same values but of different types are used in the network!)

[isaacraf95]

@zhenhuaw-me ok, i got it! I'll do it, thanks a lot!