An inference error occurred after converting ONNX to tensorrt

[wangbiao0]

When I was using UNet for image segmentation, after converting the pytorch format model to the tensorrt format, there would be a large error in the reasoning, as follows:

But the label image is like the following:

However, when I perform reasoning using the.pth format or the.onnx format, I can obtain a normal result (the prediction result is similar to a label).

When converting the onnx model to tensorrt format, I tried various methods, using code and tools (trtexec and Polygraphy), and employed a wide range of precisions (FP32/FP16/int8+ calibration). All will get wrong results.

And when using Polygraphy to analyze the layers of the onnx model and the engine model, the analysis results all passed the [instruction: polygraphy run UNet.onnx --load -- outputs UNet_onnx_out.json UNet_engine_out.json --onnxrt --trt --fail-fast】.

However, I saved the output of onnx inference and tensorrt inference as.npy files and compared the two files. Indeed, there are significant differences between them. The errors at many positions exceed 0.9.

I sincerely hope to receive everyone's help！