open-mmlab
Fail to build a custom tensorrt plugin
I wrote a plugin referring to 'mmdeploy/csrc/mmdeploy/backend_ops/tensorrt/grid_sampler', and built it successfully. However, I got an error when I ran onnx2tensorrt as follows:
you can find the plugin file in here: https://github.com/cxiang26/layernormplugin4mmdeploy we write this plugin according to https://github.com/Tlntin/LayerNorm_Plugin
envronments: `2022-07-27 19:53:02,113 - mmdeploy - INFO - Environmental information 2022-07-27 19:53:02,295 - mmdeploy - INFO - sys.platform: linux 2022-07-27 19:53:02,295 - mmdeploy - INFO - Python: 3.7.13 (default, Mar 29 2022, 02:18:16) [GCC 7.5.0] 2022-07-27 19:53:02,295 - mmdeploy - INFO - CUDA available: True 2022-07-27 19:53:02,295 - mmdeploy - INFO - GPU 0: NVIDIA GeForce RTX 3090 2022-07-27 19:53:02,295 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda 2022-07-27 19:53:02,295 - mmdeploy - INFO - NVCC: Cuda compilation tools, release 11.1, V11.1.74 2022-07-27 19:53:02,295 - mmdeploy - INFO - GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 2022-07-27 19:53:02,295 - mmdeploy - INFO - PyTorch: 1.8.0 2022-07-27 19:53:02,295 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:
- GCC 7.3
- C++ Version: 201402
- Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.1
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
- CuDNN 8.0.5
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,
2022-07-27 19:53:02,296 - mmdeploy - INFO - Backend information 2022-07-27 19:53:02,578 - mmdeploy - INFO - onnxruntime: 1.11.1 ops_is_avaliable : True 2022-07-27 19:53:02,596 - mmdeploy - INFO - tensorrt: 8.2.3.0 ops_is_avaliable : True 2022-07-27 19:53:02,607 - mmdeploy - INFO - ncnn: None ops_is_avaliable : False 2022-07-27 19:53:02,608 - mmdeploy - INFO - pplnn_is_avaliable: False 2022-07-27 19:53:02,609 - mmdeploy - INFO - openvino_is_avaliable: False 2022-07-27 19:53:02,609 - mmdeploy - INFO -
2022-07-27 19:53:02,609 - mmdeploy - INFO - Codebase information 2022-07-27 19:53:02,611 - mmdeploy - INFO - mmdet: 2.24.0 2022-07-27 19:53:02,611 - mmdeploy - INFO - mmseg: 0.25.0 2022-07-27 19:53:02,611 - mmdeploy - INFO - mmcls: 0.23.1 2022-07-27 19:53:02,611 - mmdeploy - INFO - mmocr: None 2022-07-27 19:53:02,611 - mmdeploy - INFO - mmedit: None 2022-07-27 19:53:02,611 - mmdeploy - INFO - mmdet3d: 1.0.0rc3 2022-07-27 19:53:02,611 - mmdeploy - INFO - mmpose: None 2022-07-27 19:53:02,611 - mmdeploy - INFO - mmrotate: None`
I have try it as follows
cd path/to/cudnn
export CUDNN_DIR=$(pwd)
export LD_LIBRARY_PATH=${CUDNN_DIR}:$LD_LIBRARY_PATH
python onnx2tensorrt --configs ****
However, I got the same error
You have declare
void attachToContext(cudnnContext *contextCudnn, cublasContext *contextCublas, nvinfer1::IGpuAllocator *gpuAllocator) TRT_NOEXCEPT override;
void detachFromContext() TRT_NOEXCEPT override;
But no definition in the cpp.
I have delete this two lines in .hpp, but the error is still occur
File "/home/cc/Projects/Deploy/mmdeploy/mmdeploy/backend/tensorrt/init_plugins.py", line 33, in load_tensorrt_plugin
ctypes.CDLL(lib_path)
File "/home/cc/anaconda3/envs/mmdeploy/lib/python3.7/ctypes/__init__.py", line 364, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /home/cc/Projects/Deploy/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so: undefined symbol: _Z17LayerNormV2KernelI6__halfLi32ELi32ELi1EEvPT_S2_fPKS1_S4_iP11CUstream_st
2022-07-28 13:13:46,251 - mmdeploy - ERROR - `mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt` with Call id: 0 failed. exit.
Backend TkAgg is interactive backend. Turning interactive mode on.
Can you give some tips about how to implement a custom tensorrt plugin in mmdeploy? Thanks, very much.
The error undefined symbol: xxx means that you have declare a function (either added manually or added by template), but forgot to add the definition.
You can use c++filt to check the missing function name and make sure it has been compiled
c++filt _Z17LayerNormV2KernelI6__halfLi32ELi32ELi1EEvPT_S2_fPKS1_S4_iP11CUstream_st
void LayerNormV2Kernel<__half, 32, 32, 1>(__half*, __half*, float, __half const*, __half const*, int, CUstream_st*)
Close due to no reply. Please feel free to reopen the issue if the problem still exist.