onnx-tensorrt
onnx-tensorrt copied to clipboard
onnx
TypeError: pybind11::init(): factory function returned nullptr
Description
When trying to load an ONNX model from samples using following code, I get TypeError: pybind11::init(): factory function returned nullptr.
The following is a script (newfile3.py) that points to an ONNX file:
import os, onnx, onnx_tensorrt
import onnx_tensorrt.backend as backend
onnx_path = os.path.join("/home/ubuntu/my_project", "yolo.onnx")
onnx_model = onnx.load(onnx_path)
engine = backend.prepare(onnx_model, device='CUDA:0')
The file yolo.onnx was taken from the onnx_sample repository: https://github.com/onnx/models When trying to build the engine, I get the following:
[09/26/2023-19:53:25] [TRT] [W] Unable to determine GPU memory usage
[09/26/2023-19:53:25] [TRT] [W] Unable to determine GPU memory usage
[09/26/2023-19:53:25] [TRT] [W] CUDA initialization failure with error: 35. Please check your CUDA installation: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
Traceback (most recent call last):
File "/home/ubuntu/my_project/newfile3.py", line 6, in
However, I can import tensorrt, onnx and onnx_tensorrt separately fine, and when I run: /usr/src/tensorrt/bin/trtexec, I get a "PASSED" message.
I get this same message when I run the "onnx_backend_test.py" test.
Environment
TensorRT Version: tensorrt.version (in python) produces '8.6.1' ONNX-TensorRT Version / Branch: main GPU Type: ec2 G5 A10G, NVIDIA-SMI 510.47.03 Driver Version: 510.47.03 CUDA Version: 11.6 Nvidia Driver Version: 510.47.03 CUDA Version: 11.6 CUDNN Version: 8.9.5 Operating System + Version: Ubuntu 18.04 Python Version (if applicable): 3.8.12 TensorFlow + TF2ONNX Version (if applicable): not using tensorflow PyTorch Version (if applicable): 1.13 Baremetal or Container (if container which image + tag): baremetal
Relevant Files
output from nvidia-smi:
Tue Sep 26 20:02:42 2023 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 510.47.03 Driver Version: 510.47.03 CUDA Version: 11.6 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | | 0% 33C P0 61W / 300W | 247MiB / 23028MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 2300 C python3 245MiB | +-----------------------------------------------------------------------------+
Steps To Reproduce
Packages installed: cudnn-local-repo-ubuntu1804-8.9.5.29_1.0-1_amd64.deb nv-tensorrt-local-repo-ubuntu1804-8.6.1-cuda-11.8_1.0-1_amd64.deb
The following is the ~/onnx-tensorrt/build folder:
~/onnx-tensorrt/build
drwxrwxr-x 3 ubuntu ubuntu 4096 Sep 25 21:56 third_party drwxrwxr-x 3 ubuntu ubuntu 4096 Sep 25 22:08 lib -rw-r--r-- 1 root root 3828 Sep 26 19:07 CPackSourceConfig.cmake -rw-r--r-- 1 root root 3373 Sep 26 19:07 CPackConfig.cmake -rw-r--r-- 1 root root 3825 Sep 26 19:07 cmake_install.cmake lrwxrwxrwx 1 root root 24 Sep 26 19:08 libnvonnxparser.so.8 -> libnvonnxparser.so.8.6.1 lrwxrwxrwx 1 root root 20 Sep 26 19:08 libnvonnxparser.so -> libnvonnxparser.so.8 -rwxr-xr-x 1 root root 1345992 Sep 26 19:08 libnvonnxparser.so.8.6.1 -rw-r--r-- 1 root root 1819670 Sep 26 19:08 libnvonnxparser_static.a -rw-r--r-- 1 root root 22533 Sep 26 19:21 CMakeCache.txt -rw-r--r-- 1 root root 26285 Sep 26 19:21 Makefile drwxr-xr-x 9 root root 4096 Sep 26 19:51 CMakeFiles -rw-r--r-- 1 root root 149 Sep 26 19:51 install_manifest.txt drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 26 19:52 bdist.linux-x86_64
+1
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 520.56.06 Driver Version: 522.30 CUDA Version: 11.8 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:01:00.0 On | N/A |
| N/A 49C P8 19W / N/A | 1480MiB / 6144MiB | 50% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
import onnx
import onnx_tensorrt.backend as backend
import numpy as np
model = onnx.load("model.onnx")
engine = backend.prepare(model, device="CUDA")
input_data = np.random.random(size=(1, 3, 224, 224)).astype(np.float32)
output_data = engine.run(input_data)[0]
print(output_data)
print(output_data.shape)
root@5644f2ca731e:/opt/test# python3 test_onnx.py
[11/22/2023-07:29:18] [TRT] [W] Unable to determine GPU memory usage
[11/22/2023-07:29:18] [TRT] [W] Unable to determine GPU memory usage
[11/22/2023-07:29:18] [TRT] [W] CUDA initialization failure with error: 35. Please check your CUDA installation: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
Traceback (most recent call last):
File "/opt/test/test_onnx.py", line 6, in <module>
engine = backend.prepare(model, device="CUDA")
File "/usr/local/lib/python3.10/dist-packages/onnx_tensorrt-8.6.1-py3.10.egg/onnx_tensorrt/backend.py", line 238, in prepare
File "/usr/local/lib/python3.10/dist-packages/onnx_tensorrt-8.6.1-py3.10.egg/onnx_tensorrt/backend.py", line 42, in __init__
TypeError: pybind11::init(): factory function returned nullptr
