Is the TensorRT still not support while_loop?

[yjiangling]

TensorRT Version: 8.2.3.0

trtexec --onnx=./onnx_model/model.onnx --minShapes=xs:1x1120,xlen:1 --optShapes=xs:1x160000,xlen:1 --maxShapes=xs:1x480000,xlen:1 --workspace=10240 --saveEngine=./trt_model/model-FP32.plan --device=3 --verbose --buildOnly

result-FP32.txt exit

When I use trtexec to convert onnx model to tensorrt engine, it gives the following errror:

output { name: "sub_graph_ending_node_Identity__331:0" type { tensor_type { elem_type: 6 shape { dim { dim_param: "unk__3181" } } } } } output { name: "sub_graph_ending_node_Identity__332:0" type { tensor_type { elem_type: 1 shape { dim { dim_param: "unk__3182" } } } } } } type: GRAPH } domain: ""

[12/26/2022-03:34:40] [E] [TRT] parsers/onnx/ModelImporter.cpp:783: --- End node --- [12/26/2022-03:34:40] [E] [TRT] parsers/onnx/ModelImporter.cpp:785: ERROR: parsers/onnx/builtin_op_importers.cpp:4870 In function importFallbackPluginImporter: [8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?" [12/26/2022-03:34:40] [E] Failed to parse onnx file [12/26/2022-03:34:40] [E] Parsing model failed [12/26/2022-03:34:40] [E] Failed to create engine from model. [12/26/2022-03:34:40] [E] Engine set up failed

What's wrong with it? I have no ideal to deal with it? Anyone can give me some help? Thanks a lot! By the way, the model is Conformer with while_loop operator to achieve auto-regressive decoding.

[yjiangling]

https://github.com/yjiangling/Temp_Model/releases/download/1.0/model.zip

Here is the address to get the ONNX model.

[yjiangling]

The error have been solved, but a new error occured as below:

[01/03/2023-08:03:16] [E] [TRT] parsers/onnx/ModelImporter.cpp:783: --- End node --- [01/03/2023-08:03:16] [E] [TRT] parsers/onnx/ModelImporter.cpp:785: ERROR: parsers/onnx/ModelImporter.cpp:166 In function parseGraph: [6] Invalid Node - generic_loop_Loop__352 [graphShapeAnalyzer.cpp::processCheck::581] Error Code 4: Internal Error ((Unnamed Layer* 3580) [Recurrence]: inputs to IRecurrenceLayer have different dimensions. First input has dimensions [3,1] and second input has dimensions [3,2]. ) [graphShapeAnalyzer.cpp::processCheck::581] Error Code 4: Internal Error ((Unnamed Layer* 3580) [Recurrence]: inputs to IRecurrenceLayer have different dimensions. First input has dimensions [3,1] and second input has dimensions [3,2]. ) [graphShapeAnalyzer.cpp::processCheck::581] Error Code 4: Internal Error ((Unnamed Layer* 3580) [Recurrence]: inputs to IRecurrenceLayer have different dimensions. First input has dimensions [3,1] and second input has dimensions [3,2]. ) [01/03/2023-08:03:16] [E] Failed to parse onnx file [01/03/2023-08:03:16] [E] Parsing model failed [01/03/2023-08:03:16] [E] Failed to create engine from model. [01/03/2023-08:03:16] [E] Engine set up failed

Anyone can give some help or suggestion? Thanks a lot!

[zhenhuaw-me]

@yjiangling As the error tips, there might be a dimension conflict. Could you please check if the model is correct and try out TensorRT 8.6 release?