mmdeploy icon indicating copy to clipboard operation
mmdeploy copied to clipboard

Published 20 hours ago • verified publisher icon open-mmlab

mmdeploy - ERROR - `mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt` with Call id: 1 failed. exit.

Open Richardych opened this issue 2 years ago • 5 comments

When I convert a mmrotate model to trt, I got the following error. I have successfully build the tensorRT converter, and there exist the file : mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so

08/30/2022-23:06:21] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 241 MiB, GPU 363 MiB [08/30/2022-23:06:21] [TRT] [W] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively sup port INT64. Attempting to cast down to INT32. [08/30/2022-23:06:21] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:25] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:25] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:25] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:25] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:25] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:25] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:27] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:27] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:27] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:27] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [08/30/2022-23:06:29] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin. [08/30/2022-23:06:29] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: Process Process-3: Traceback (most recent call last): File "/home/huakun.ych/anaconda3/envs/mmdeploy/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/huakun.ych/anaconda3/envs/mmdeploy/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/huakun.ych/anaconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__ ret = func(*args, **kwargs) File "/home/huakun.ych/anaconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 79, in onnx2tensorrt from_onnx( File "/home/huakun.ych/anaconda3/envs/mmdeploy/lib/python3.8/site-packages/mmdeploy/backend/tensorrt/utils.py", line 113, in from_onnx raise RuntimeError(f'Failed to parse onnx, {error_msgs}') RuntimeError: Failed to parse onnx, In node 1369 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"

Richardych avatar Aug 30 '22 15:08 Richardych

@lvhan028

tpoisonooo avatar Aug 30 '22 15:08 tpoisonooo

when I run "python tools/check_env.py", I got : ` 022-08-30 23:34:21,649 - mmdeploy - INFO - Environmental information [29/1904] 2022-08-30 23:34:21,873 - mmdeploy - INFO - sys.platform: linux 2022-08-30 23:34:21,873 - mmdeploy - INFO - Python: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:04:18) [GCC 10.3.0] 2022-08-30 23:34:21,873 - mmdeploy - INFO - CUDA available: True 2022-08-30 23:34:21,873 - mmdeploy - INFO - GPU 0,1,2,3,4,5,6,7: Tesla P100-PCIE-16GB 2022-08-30 23:34:21,873 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda 2022-08-30 23:34:21,873 - mmdeploy - INFO - NVCC: Cuda compilation tools, release 10.1, V10.1.24 2022-08-30 23:34:21,873 - mmdeploy - INFO - GCC: gcc (GCC) 6.4.0 20170704 (Red Hat 6.4.0-1) 2022-08-30 23:34:21,873 - mmdeploy - INFO - PyTorch: 1.8.0 2022-08-30 23:34:21,873 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:

  • GCC 7.3
  • C++ Version: 201402
  • Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
  • Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
  • OpenMP 201511 (a.k.a. OpenMP 4.5)
  • NNPACK is enabled
  • CPU capability usage: AVX2
  • CUDA Runtime 10.2
  • NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;a rch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
  • CuDNN 7.6.5
  • Magma 2.5.2
  • Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QN NPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds - Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redu ndant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errn o -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1 .8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK =ON, USE_OPENMP=ON, 022-08-30 23:34:21,873 - mmdeploy - INFO - TorchVision: 0.9.0 2022-08-30 23:34:21,873 - mmdeploy - INFO - OpenCV: 4.6.0 2022-08-30 23:34:21,873 - mmdeploy - INFO - MMCV: 1.6.0 2022-08-30 23:34:21,874 - mmdeploy - INFO - MMCV Compiler: GCC 7.3 2022-08-30 23:34:21,874 - mmdeploy - INFO - MMCV CUDA Compiler: 10.2 2022-08-30 23:34:21,874 - mmdeploy - INFO - MMDeploy: 0.7.0+638e33a 2022-08-30 23:34:21,874 - mmdeploy - INFO -

2022-08-30 23:34:21,874 - mmdeploy - INFO - Backend information 2022-08-30 23:34:22,375 - mmdeploy - INFO - onnxruntime: 1.8.1 ops_is_avaliable : True 2022-08-30 23:34:22,407 - mmdeploy - INFO - tensorrt: 8.2.3.0 ops_is_avaliable : False 2022-08-30 23:34:22,425 - mmdeploy - INFO - ncnn: None ops_is_avaliable : False 2022-08-30 23:34:22,425 - mmdeploy - INFO - pplnn_is_avaliable: False 2022-08-30 23:34:22,426 - mmdeploy - INFO - openvino_is_avaliable: False 2022-08-30 23:34:22,443 - mmdeploy - INFO - snpe_is_available: False 2022-08-30 23:34:22,443 - mmdeploy - INFO -

2022-08-30 23:34:22,443 - mmdeploy - INFO - Codebase information 2022-08-30 23:34:24,002 - mmdeploy - INFO - mmdet: 2.25.1 2022-08-30 23:34:24,002 - mmdeploy - INFO - mmseg: None 2022-08-30 23:34:24,003 - mmdeploy - INFO - mmcls: None 2022-08-30 23:34:24,003 - mmdeploy - INFO - mmocr: None 2022-08-30 23:34:24,003 - mmdeploy - INFO - mmedit: None 2022-08-30 23:34:24,003 - mmdeploy - INFO - mmdet3d: None 2022-08-30 23:34:24,003 - mmdeploy - INFO - mmpose: None 2022-08-30 23:34:24,003 - mmdeploy - INFO - mmrotate: 0.3.2 `

Richardych avatar Aug 30 '22 15:08 Richardych

hello ,when l try to convert the model of yolov3 from mmdetection to tensorRT in docker,l meet the same error

root@077b3c4bf096:~/workspace/mmdeploy# python ./tools/deploy.py     configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py     /root/workspace/mmdetection/configs/yolo/yolov3_d53_mstrain-608_273e_coco.py     /root/workspace/mmdetection/checkpoints/yolo/yolov3_d53_mstrain-608_273e_coco_20210518_115020-a2c3acb8.pth     /root/workspace/mmdetection/demo/demo.jpg     --work-dir work_dir     --show     --device cuda:0
[2022-08-31 07:48:42.403] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
[2022-08-31 07:48:44.003] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
[2022-08-31 07:48:45.620] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
2022-08-31 07:48:45,625 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
load checkpoint from local path: /root/workspace/mmdetection/checkpoints/yolo/yolov3_d53_mstrain-608_273e_coco_20210518_115020-a2c3acb8.pth
/root/workspace/mmdetection/mmdet/datasets/utils.py:66: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  warnings.warn(
2022-08-31 07:48:49,454 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
2022-08-31 07:48:49,454 - mmdeploy - INFO - Export PyTorch model to ONNX: work_dir/end2end.onnx.
2022-08-31 07:48:49,522 - mmdeploy - WARNING - Can not find torch._C._jit_pass_onnx_deduplicate_initializers, function rewrite will not be applied
/root/workspace/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:3631: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn(
/root/workspace/mmdetection/mmdet/models/dense_heads/yolo_head.py:127: UserWarning: DeprecationWarning: `anchor_generator` is deprecated, please use "prior_generator" instead
  warnings.warn('DeprecationWarning: `anchor_generator` is deprecated, '
/root/workspace/mmdetection/mmdet/core/anchor/anchor_generator.py:333: UserWarning: ``grid_anchors`` would be deprecated soon. Please use ``grid_priors`` 
  warnings.warn('``grid_anchors`` would be deprecated soon. '
/root/workspace/mmdetection/mmdet/core/anchor/anchor_generator.py:369: UserWarning: ``single_level_grid_anchors`` would be deprecated soon. Please use ``single_level_grid_priors`` 
  warnings.warn(
/root/workspace/mmdetection/mmdet/core/bbox/coder/yolo_bbox_coder.py:73: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(-1) == bboxes.size(-1) == 4
/root/workspace/mmdeploy/mmdeploy/pytorch/functions/topk.py:56: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if k > size:
/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:259: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
/root/workspace/mmdeploy/mmdeploy/mmcv/ops/nms.py:178: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  out_boxes = min(num_boxes, after_topk)
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
2022-08-31 07:48:57,157 - mmdeploy - INFO - Execute onnx optimize passes.
2022-08-31 07:48:58,336 - mmdeploy - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
[2022-08-31 07:49:00.457] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel'
2022-08-31 07:49:00,461 - mmdeploy - INFO - Start pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt in subprocess
2022-08-31 07:49:00,603 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
[08/31/2022-07:49:00] [TRT] [I] [MemUsageChange] Init CUDA: CPU +313, GPU +0, now: CPU 405, GPU 220 (MiB)
[08/31/2022-07:49:01] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 405 MiB, GPU 220 MiB
[08/31/2022-07:49:01] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 540 MiB, GPU 254 MiB
[08/31/2022-07:49:01] [TRT] [W] parsers/onnx/onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[08/31/2022-07:49:02] [TRT] [W] parsers/onnx/onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[08/31/2022-07:49:03] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[08/31/2022-07:49:03] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: 
[08/31/2022-07:49:03] [TRT] [I] Successfully created plugin: TRTBatchedNMS
[08/31/2022-07:49:04] [TRT] [W] TensorRT was linked against cuBLAS/cuBLASLt 11.6.5 but loaded cuBLAS/cuBLASLt 11.5.1
[08/31/2022-07:49:04] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +483, GPU +206, now: CPU 1500, GPU 460 (MiB)
[08/31/2022-07:49:04] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +115, GPU +54, now: CPU 1615, GPU 514 (MiB)
[08/31/2022-07:49:04] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[08/31/2022-07:50:00] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[08/31/2022-07:50:13] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are: 
[08/31/2022-07:50:13] [TRT] [W]  (# 3 (SHAPE input))
[08/31/2022-07:50:13] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[08/31/2022-07:50:13] [TRT] [W]  (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00)
[08/31/2022-07:50:13] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[08/31/2022-07:50:13] [TRT] [W]  (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00)
[08/31/2022-07:50:13] [TRT] [W]  (# 2 (SHAPE input))
[08/31/2022-07:50:13] [TRT] [W] Skipping tactic 0 due to insuficient memory on requested size of 2442276864 detected for tactic 0.
[08/31/2022-07:50:13] [TRT] [E] 10: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[971_57...Concat_508]}.)
Process Process-3:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/root/workspace/mmdeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 79, in onnx2tensorrt
    from_onnx(
  File "/root/workspace/mmdeploy/mmdeploy/backend/tensorrt/utils.py", line 153, in from_onnx
    assert engine is not None, 'Failed to create TensorRT engine'
AssertionError: Failed to create TensorRT engine
2022-08-31 07:50:14,174 - mmdeploy - ERROR - `mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt` with Call id: 1 failed. exit.

when I run "python tools/check_env.py", I got :

2022-08-31 07:52:19,401 - mmdeploy - INFO - 

2022-08-31 07:52:19,402 - mmdeploy - INFO - **********Environmental information**********
2022-08-31 07:52:19,557 - mmdeploy - INFO - sys.platform: linux
2022-08-31 07:52:19,557 - mmdeploy - INFO - Python: 3.8.13 (default, Mar 28 2022, 11:38:47) [GCC 7.5.0]
2022-08-31 07:52:19,557 - mmdeploy - INFO - CUDA available: True
2022-08-31 07:52:19,557 - mmdeploy - INFO - GPU 0: NVIDIA GeForce GTX 1660 Ti
2022-08-31 07:52:19,557 - mmdeploy - INFO - GPU 1: Tesla M40 24GB
2022-08-31 07:52:19,558 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda
2022-08-31 07:52:19,558 - mmdeploy - INFO - NVCC: Cuda compilation tools, release 11.6, V11.6.124
2022-08-31 07:52:19,558 - mmdeploy - INFO - GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
2022-08-31 07:52:19,558 - mmdeploy - INFO - PyTorch: 1.10.0
2022-08-31 07:52:19,558 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.2
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, 

2022-08-31 07:52:19,558 - mmdeploy - INFO - TorchVision: 0.11.0
2022-08-31 07:52:19,558 - mmdeploy - INFO - OpenCV: 4.6.0
2022-08-31 07:52:19,558 - mmdeploy - INFO - MMCV: 1.5.3
2022-08-31 07:52:19,558 - mmdeploy - INFO - MMCV Compiler: GCC 7.3
2022-08-31 07:52:19,558 - mmdeploy - INFO - MMCV CUDA Compiler: 11.3
2022-08-31 07:52:19,558 - mmdeploy - INFO - MMDeploy: 0.7.0+1f8d889
2022-08-31 07:52:19,558 - mmdeploy - INFO - 

2022-08-31 07:52:19,558 - mmdeploy - INFO - **********Backend information**********
2022-08-31 07:52:20,059 - mmdeploy - INFO - onnxruntime: 1.8.1  ops_is_avaliable : True
2022-08-31 07:52:20,089 - mmdeploy - INFO - tensorrt: 8.2.4.2   ops_is_avaliable : True
2022-08-31 07:52:20,108 - mmdeploy - INFO - ncnn: None  ops_is_avaliable : False
2022-08-31 07:52:20,110 - mmdeploy - INFO - pplnn_is_avaliable: False
2022-08-31 07:52:20,112 - mmdeploy - INFO - openvino_is_avaliable: False
2022-08-31 07:52:20,130 - mmdeploy - INFO - snpe_is_available: False
2022-08-31 07:52:20,130 - mmdeploy - INFO - 

2022-08-31 07:52:20,130 - mmdeploy - INFO - **********Codebase information**********
2022-08-31 07:52:20,131 - mmdeploy - INFO - mmdet:      2.25.1
2022-08-31 07:52:20,131 - mmdeploy - INFO - mmseg:      None
2022-08-31 07:52:20,131 - mmdeploy - INFO - mmcls:      None
2022-08-31 07:52:20,132 - mmdeploy - INFO - mmocr:      None
2022-08-31 07:52:20,132 - mmdeploy - INFO - mmedit:     None
2022-08-31 07:52:20,132 - mmdeploy - INFO - mmdet3d:    None
2022-08-31 07:52:20,132 - mmdeploy - INFO - mmpose:     None
2022-08-31 07:52:20,132 - mmdeploy - INFO - mmrotate:   None

tcexeexe avatar Aug 31 '22 07:08 tcexeexe

l have solve the problem by update the nvidia driver 470 to 510. Share it with others who have the same problem

tcexeexe avatar Sep 04 '22 08:09 tcexeexe

Hi, @Richardych #947 has fixed crash issue when converting tensorrt model on cuda10.2 platform. Could you update mmdeploy to the latest version and try it again?

lvhan028 avatar Sep 05 '22 09:09 lvhan028

l have solve the problem by update the nvidia driver 470 to 510. Share it with others who have the same problem

Thanks for your solution. But I still meet the same error on converting yolov3 from mmdetection to tensorrt.

converting code and error

python -W ignore /root/workspace/mmdeploy/tools/deploy.py \ /root/workspace/mmdeploy/configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py \ /mmdetection/configs/yolo/yolov3_d53_mstrain-608_273e_coco.py \ /root/workspace/temp/checkpoints/yolov3.pth \ /root/workspace/temp/testdata/0000008_01999_d_0000040.jpg \ --work-dir /root/workspace/temp/tensorrt_models/yolov3 \ --device cuda:0 Log 2022-11-27 22:47:34,391 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess load checkpoint from local path: /root/workspace/temp/checkpoints/yolov3.pth 2022-11-27 22:47:38,420 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 2022-11-27 22:47:38,420 - mmdeploy - INFO - Export PyTorch model to ONNX: /root/workspace/temp/tensorrt_models/yolov3/end2end.onnx. 2022-11-27 22:47:38,450 - mmdeploy - WARNING - Can not find torch._C._jit_pass_onnx_deduplicate_initializers, function rewrite will not be applied WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. 2022-11-27 22:47:46,218 - mmdeploy - INFO - Execute onnx optimize passes. 2022-11-27 22:47:47,626 - mmdeploy - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx 2022-11-27 22:47:49,642 - mmdeploy - INFO - Start pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt in subprocess 2022-11-27 22:47:49,773 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so [11/27/2022-22:47:49] [TRT] [I] [MemUsageChange] Init CUDA: CPU +172, GPU +0, now: CPU 258, GPU 642 (MiB) [11/27/2022-22:47:50] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 258 MiB, GPU 642 MiB [11/27/2022-22:47:50] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 321 MiB, GPU 642 MiB [11/27/2022-22:47:50] [TRT] [W] parsers/onnx/onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. [11/27/2022-22:47:51] [TRT] [W] parsers/onnx/onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped [11/27/2022-22:47:52] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin. [11/27/2022-22:47:52] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: [11/27/2022-22:47:52] [TRT] [I] Successfully created plugin: TRTBatchedNMS [11/27/2022-22:47:53] [TRT] [W] TensorRT was linked against cuBLAS/cuBLASLt 11.6.5 but loaded cuBLAS/cuBLASLt 11.5.1 [11/27/2022-22:47:53] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +229, GPU +94, now: CPU 1028, GPU 736 (MiB) [11/27/2022-22:47:53] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +112, GPU +44, now: CPU 1140, GPU 780 (MiB) [11/27/2022-22:47:53] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored. [11/27/2022-22:48:16] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [11/27/2022-22:48:28] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are: [11/27/2022-22:48:28] [TRT] [W] (# 3 (SHAPE input)) [11/27/2022-22:48:28] [TRT] [W] (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00) [11/27/2022-22:48:28] [TRT] [W] (ONNX_RESIZE (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00) 2.000000e+00) [11/27/2022-22:48:28] [TRT] [W] (# 2 (SHAPE input)) [11/27/2022-22:48:28] [TRT] [W] (ONNX_RESIZE (+ (CEIL_DIV (+ (# 2 (SHAPE input)) -32) 32) 1) 2.000000e+00) [11/27/2022-22:48:28] [TRT] [W] (ONNX_RESIZE (+ (CEIL_DIV (+ (# 3 (SHAPE input)) -32) 32) 1) 2.000000e+00) [11/27/2022-22:48:28] [TRT] [W] Skipping tactic 0 due to insuficient memory on requested size of 2442276864 detected for tactic 0. [11/27/2022-22:48:28] [TRT] [E] 10: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[971_57...Concat_510]}.) Process Process-3: Traceback (most recent call last): File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/root/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__ ret = func(*args, **kwargs) File "/root/workspace/mmdeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 79, in onnx2tensorrt from_onnx( File "/root/workspace/mmdeploy/mmdeploy/backend/tensorrt/utils.py", line 226, in from_onnx assert engine is not None, 'Failed to create TensorRT engine' AssertionError: Failed to create TensorRT engine 2022-11-27 22:48:29,203 - mmdeploy - ERROR - mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt with Call id: 1 failed. exit.

Check my environment

dpkg –l | grep nvinfer image python ./mmdeploy/tools/check_env.py image

Inspect the ONNX model with Polygraphy (It works)

Refer to polygraph POLYGRAPHY_AUTOINSTALL_DEPS=1 polygraphy inspect model /models/end2end.onnx --show layers --display-as=trt --plugins ./libmmdeploy_tensorrt_ops.so image

Jason-cs18 avatar Nov 27 '22 23:11 Jason-cs18

@lvhan028 @tpoisonooo @tcexeexe

Jason-cs18 avatar Nov 27 '22 23:11 Jason-cs18

Try to use this deployment config detection_tensorrt_dynamic-64x64-608x608.py, since the maximum input resolution of yolov3 from mmdet is img_scale=(320, 320)

lvhan028 avatar Dec 05 '22 03:12 lvhan028

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

github-actions[bot] avatar Dec 23 '22 01:12 github-actions[bot]

This issue is closed because it has been stale for 5 days. Please open a new issue if you have similar issues or you have any new updates now.

github-actions[bot] avatar Apr 01 '23 01:04 github-actions[bot]