Segmentation Fault failure of TensorRT 8.6.1 when converting onnx on GPU GeForce RTX 3050 Ti

[steve-volley]

Description

I tried to convert my onnx model to .trt but trtexec segfaulted. See attached log output of trtexec ... the program segfaults after the final line you see in that file.

Environment

TensorRT Version: 8.6.1

NVIDIA GPU: GeForce RTX 3050 Ti

NVIDIA Driver Version: 530.30.02

CUDA Version: 12.1

CUDNN Version: 8.9.7

Operating System: Ubuntu 20.04 (Linux 5.15.0-94-generic #104~20.04.1-Ubuntu SMP Tue Jan 16 13:34:09 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux)

Python Version (if applicable):

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

ONNX link: https://drive.google.com/file/d/1rYr_jHkqa7Y9dPYTSNMtaWYbSwF88FdS/view?usp=sharing

Verbose output log: https://drive.google.com/file/d/1OeaYb0iZETvG7OOkxUQqO7rFuc2JEIdC/view?usp=sharing

Steps To Reproduce

Commands or scripts: /usr/src/tensorrt/bin/trtexec --verbose --onnx=./models_onnx_reid_osnet_ibn_ms_m_c_dynamic.onnx --saveEngine=output.trt

Have you tried the latest release?: Yes, this is it

[zerollzeng]

I didn't reproduce the issue on TRT 8.6 and 9.2. Could you please:

1. try converting it in our official tensorrt container?
2. try trt 9.2/9.3 Thanks!

[zerollzeng]

[02/24/2024-15:17:52] [I] Latency: min = 0.954285 ms, max = 1.15598 ms, mean = 0.957755 ms, median = 0.95752 ms, percentile(90%) = 0.959534 ms, percentile(95%) = 0.960541 ms, percentile(99%) = 0.963623 ms
[02/24/2024-15:17:52] [I] Enqueue Time: min = 0.541138 ms, max = 1.26636 ms, mean = 0.72002 ms, median = 0.709229 ms, percentile(90%) = 0.869873 ms, percentile(95%) = 0.879272 ms, percentile(99%) = 0.920654 ms
[02/24/2024-15:17:52] [I] H2D Latency: min = 0.0183105 ms, max = 0.0332794 ms, mean = 0.0194599 ms, median = 0.0192871 ms, percentile(90%) = 0.020874 ms, percentile(95%) = 0.0218201 ms, percentile(99%) = 0.0240631 ms
[02/24/2024-15:17:52] [I] GPU Compute Time: min = 0.931763 ms, max = 1.13049 ms, mean = 0.934883 ms, median = 0.934875 ms, percentile(90%) = 0.936035 ms, percentile(95%) = 0.936035 ms, percentile(99%) = 0.937012 ms
[02/24/2024-15:17:52] [I] D2H Latency: min = 0.00268555 ms, max = 0.00775146 ms, mean = 0.00341897 ms, median = 0.00317383 ms, percentile(90%) = 0.00439453 ms, percentile(95%) = 0.00463867 ms, percentile(99%) = 0.00488281 ms
[02/24/2024-15:17:52] [I] Total Host Walltime: 3.00328 s
[02/24/2024-15:17:52] [I] Total GPU Compute Time: 2.9963 s
[02/24/2024-15:17:52] [I] Explanations of the performance metrics are printed in the verbose logs.
[02/24/2024-15:17:52] [I] 
&&&& PASSED TensorRT.trtexec [TensorRT v8601] # trtexec --onnx=models_onnx_reid_osnet_ibn_ms_m_c_dynamic.onnx

[ttyio]

closing since no activity for more than 3 weeks, pls reopen if you still have question, thanks!