mmdeploy icon indicating copy to clipboard operation
mmdeploy copied to clipboard
verified publisher icon open-mmlab

[Bug] RTMDet : Error when deploying onnx/engine model using tensorRT backend configs

Open SushmaDG opened this issue 1 year ago • 11 comments

Checklist

  • [X] I have searched related issues but cannot get the expected help.
  • [X] 2. I have read the FAQ documentation but cannot get the expected help.
  • [X] 3. The bug has not been fixed in the latest version.

Describe the bug

I am trying to deploy pytorch model to a backend tensorrt model using tools/deploy.py as below:

python mmdeploy/tools/deploy.py \
    mmdeploy\configs\mmdet\detection\detection_tensorrt_static-640x640.py \
    mmdetection\rtmdet_s_8xb32-300e_coco.py \
    mmdetection\rtmdet_s_8xb32-300e_coco_20220905_161602-387a891e.pth \
    mmdetection/demo/demo.jpg \
    --work-dir mmdeploy_model\rtmdet\trt \
    --device cuda \
    --dump-info

The checkpoint was downloaded from the link ->

wget -P checkpoint https://download.openmmlab.com/mmdetection/v3.0/rtmdet/rtmdet_s_8xb32-300e_coco/rtmdet_s_8xb32-300e_coco_20220905_161602-387a891e.pth

Error Traceback

07/04 17:43:16 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
07/04 17:43:18 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
07/04 17:43:18 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: mmdetection\rtmdet_s_8xb32-300e_coco_20220905_161602-387a891e.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: data_preprocessor.mean, data_preprocessor.std

07/04 17:43:19 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
07/04 17:43:19 - mmengine - INFO - Export PyTorch model to ONNX: mmdeploy\mmdeploy_models\mmdet\trt\end2end.onnx.
07/04 17:43:23 - mmengine - WARNING - Can not find torch.nn.functional._scaled_dot_product_attention, function rewrite will not be applied
07/04 17:43:23 - mmengine - WARNING - Can not find mmdet.models.utils.transformer.PatchMerging.forward, function rewrite will not be applied
C:\Users\.conda\envs\openmmlab2\lib\site-packages\mmdeploy\codebase\mmdet\models\detectors\single_stage.py:80: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  img_shape = [int(val) for val in img_shape]
C:\Users\.conda\envs\openmmlab2\lib\site-packages\mmdeploy\codebase\mmdet\models\detectors\single_stage.py:80: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  img_shape = [int(val) for val in img_shape]
C:\Users\.conda\envs\openmmlab2\lib\site-packages\mmdeploy\core\optimizers\function_marker.py:161: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in shape_)
C:\Users\.conda\envs\openmmlab2\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:3527.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
C:\Users\.conda\envs\openmmlab2\lib\site-packages\mmdeploy\mmcv\ops\nms.py:477: TracerWarning: Converting a tensor to a NumPy array might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  int(scores.numpy().shape[-1]),
C:\Users\.conda\envs\openmmlab2\lib\site-packages\mmdeploy\mmcv\ops\nms.py:148: TracerWarning: Converting a tensor to a NumPy array might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  num_boxes = num_boxes.numpy()
07/04 17:43:29 - mmengine - ERROR - C:\Users\.conda\envs\openmmlab2\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py - pop_mp_output - 80 - `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed. exit.

Reproduction

  • cd ${PATH_TO_MMDEPLOY}

  • get the RTMDet-s checkpoint

wget -P checkpoint https://download.openmmlab.com/mmdetection/v3.0/rtmdet/rtmdet_s_8xb32-300e_coco/rtmdet_s_8xb32-300e_coco_20220905_161602-387a891e.pth
  • command for model conversion
python mmdeploy/tools/deploy.py \
    mmdeploy\configs\mmdet\detection\detection_tensorrt_static-640x640.py \
    mmdetection\rtmdet_s_8xb32-300e_coco.py \
    mmdetection\rtmdet_s_8xb32-300e_coco_20220905_161602-387a891e.pth \
    mmdetection/demo/demo.jpg \
    --work-dir mmdeploy_model\rtmdet\trt \
    --device cuda \
    --dump-info

Environment

I get the following on running `python tools/check_env.py`


07/04 17:29:10 - mmengine - INFO -

07/04 17:29:10 - mmengine - INFO - **********Environmental information**********
07/04 17:29:33 - mmengine - INFO - sys.platform: win32
07/04 17:29:33 - mmengine - INFO - Python: 3.9.19 (main, May  6 2024, 20:12:36) [MSC v.1916 64 bit (AMD64)]
07/04 17:29:33 - mmengine - INFO - CUDA available: True
07/04 17:29:33 - mmengine - INFO - MUSA available: False
07/04 17:29:33 - mmengine - INFO - numpy_random_seed: 2147483648
07/04 17:29:33 - mmengine - INFO - GPU 0: NVIDIA GeForce RTX 3050 Ti Laptop GPU
07/04 17:29:33 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
07/04 17:29:33 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.8, V11.8.89
07/04 17:29:33 - mmengine - INFO - MSVC: Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30147 for x64
07/04 17:29:33 - mmengine - INFO - GCC: n/a
07/04 17:29:33 - mmengine - INFO - PyTorch: 2.1.2+cu118
07/04 17:29:33 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - C++ Version: 199711
  - MSVC 192930151
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)
  - OpenMP 2019
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX512
  - CUDA Runtime 11.8
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.7
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=C:/actions-runner/_work/pytorch/pytorch/builder/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /bigobj /FS -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /utf-8 /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.1.2, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

07/04 17:29:33 - mmengine - INFO - TorchVision: 0.16.2+cu118
07/04 17:29:33 - mmengine - INFO - OpenCV: 4.10.0
07/04 17:29:33 - mmengine - INFO - MMEngine: 0.10.4
07/04 17:29:33 - mmengine - INFO - MMCV: 2.1.0
07/04 17:29:33 - mmengine - INFO - MMCV Compiler: MSVC 192930148
07/04 17:29:33 - mmengine - INFO - MMCV CUDA Compiler: 11.8
07/04 17:29:33 - mmengine - INFO - MMDeploy: 1.3.1+5a3be94
07/04 17:29:33 - mmengine - INFO -

07/04 17:29:33 - mmengine - INFO - **********Backend information**********
07/04 17:29:34 - mmengine - INFO - tensorrt:    8.6.1
07/04 17:29:34 - mmengine - INFO - tensorrt custom ops: Available
07/04 17:29:36 - mmengine - INFO - ONNXRuntime: 1.8.1
07/04 17:29:36 - mmengine - INFO - ONNXRuntime-gpu:     1.18.1
07/04 17:29:36 - mmengine - INFO - ONNXRuntime custom ops:      Available
07/04 17:29:36 - mmengine - INFO - pplnn:       None
07/04 17:29:36 - mmengine - INFO - ncnn:        None
07/04 17:29:36 - mmengine - INFO - snpe:        None
07/04 17:29:36 - mmengine - INFO - openvino:    None
07/04 17:29:36 - mmengine - INFO - torchscript: 2.1.2+cu118
07/04 17:29:36 - mmengine - INFO - torchscript custom ops:      NotAvailable
07/04 17:29:36 - mmengine - INFO - rknn-toolkit:        None
07/04 17:29:36 - mmengine - INFO - rknn-toolkit2:       None
07/04 17:29:36 - mmengine - INFO - ascend:      None
07/04 17:29:36 - mmengine - INFO - coreml:      None
07/04 17:29:36 - mmengine - INFO - tvm: None
07/04 17:29:36 - mmengine - INFO - vacc:        None
07/04 17:29:36 - mmengine - INFO -

07/04 17:29:36 - mmengine - INFO - **********Codebase information**********
07/04 17:29:36 - mmengine - INFO - mmdet:       3.3.0
07/04 17:29:36 - mmengine - INFO - mmseg:       None
07/04 17:29:36 - mmengine - INFO - mmpretrain:  None
07/04 17:29:36 - mmengine - INFO - mmocr:       None
07/04 17:29:36 - mmengine - INFO - mmagic:      None
07/04 17:29:36 - mmengine - INFO - mmdet3d:     None
07/04 17:29:36 - mmengine - INFO - mmpose:      None
07/04 17:29:36 - mmengine - INFO - mmrotate:    None
07/04 17:29:36 - mmengine - INFO - mmaction:    None
07/04 17:29:36 - mmengine - INFO - mmrazor:     None
07/04 17:29:36 - mmengine - INFO - mmyolo:      None

Additional information:

  • I have built the tensorrt custom ops following the documentation -> https://github.com/open-mmlab/mmdeploy/blob/main/docs/en/01-how-to-build/windows.md
  • The included mmdetection is the latest version => v3.3.0

Any help regarding this bug is appreciated. Thank you

SushmaDG avatar Jul 04 '24 16:07 SushmaDG

detection_tensorrt_static-640x640.py换成dynamic320x320-1344x1344的试试

ljl02521 avatar Jul 12 '24 08:07 ljl02521

detection_tensorrt_static-640x640.py换成dynamic320x320-1344x1344的试试

I get the same error:

mmengine - ERROR - C:\Users\.conda\envs\openmmlab2\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py 
- pop_mp_output - 80 - `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed. exit.

SushmaDG avatar Jul 12 '24 08:07 SushmaDG

detection_tensorrt_static-640x640.py换成dynamic320x320-1344x1344的试试

I get the same error:

mmengine - ERROR - C:\Users\.conda\envs\openmmlab2\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py 
- pop_mp_output - 80 - `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed. exit.

I was answering your question.

ljl02521 avatar Jul 12 '24 08:07 ljl02521

I encountered the same issue. Has it been resolved?

chemingmei avatar Dec 02 '24 07:12 chemingmei

I encountered the same issue. Has it been resolved?

No, it hasn't been resolved. Tried with different input sizes, both static and dynamic, but I get the same error!

SushmaDG avatar Dec 02 '24 07:12 SushmaDG

https://github.com/open-mmlab/mmdeploy/issues/2819 He solved this problem by pip install onnxruntime==1.16.0, I tried it and it worked, but there will be new problems in model inference [2024-12-02 21:55:54.796] [mmdeploy] [error] [ort_net.cpp:205] unhandled exception when creating ORTNet: D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1131 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 1114 "" when trying to load "C:\Users\ouc.conda\envs\gko\lib\site-packages\mmdeploy_runtime\onnxruntime_providers_cuda.dll"

My cuda version is the same as yours, maybe you can also try it, if it solves it it would be great, if there is an error in the subsequent run like me, it proves that this method is invalid. Looking forward to your feedback!

chemingmei avatar Dec 03 '24 07:12 chemingmei

#2819 He solved this problem by pip install onnxruntime==1.16.0, I tried it and it worked, but there will be new problems in model inference [2024-12-02 21:55:54.796] [mmdeploy] [error] [ort_net.cpp:205] unhandled exception when creating ORTNet: D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1131 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 1114 "" when trying to load "C:\Users\ouc.conda\envs\gko\lib\site-packages\mmdeploy_runtime\onnxruntime_providers_cuda.dll"

My cuda version is the same as yours, maybe you can also try it, if it solves it it would be great, if there is an error in the subsequent run like me, it proves that this method is invalid. Looking forward to your feedback!

Unfortunately, I do not have that environment anymore, so it will take a while to reproduce it. I will try with onnxruntime=1.16.0 and let you know!

SushmaDG avatar Dec 03 '24 16:12 SushmaDG

I rebuilt the environment and successfully resolved the TensorRT version deployment issue. I suspect the previous problem occurred because I deployed the ONNX version (CPU) first, and then deployed TensorRT, which caused some package call errors. I think this may have been due to having both onnxruntime and onnxruntime-gpu installed at the same time, but I'm not entirely sure if that was the cause. After reinstalling, everything is running fine.

chemingmei avatar Dec 04 '24 00:12 chemingmei

I rebuilt the environment and successfully resolved the TensorRT version deployment issue. I suspect the previous problem occurred because I deployed the ONNX version (CPU) first, and then deployed TensorRT, which caused some package call errors. I think this may have been due to having both onnxruntime and onnxruntime-gpu installed at the same time, but I'm not entirely sure if that was the cause. After reinstalling, everything is running fine.

That's great to hear! Could you please share the output of "python tools/check_env.py"? Its just to understand if there are some version mismatch with my environment

SushmaDG avatar Dec 04 '24 10:12 SushmaDG

Of course, I hope it can help you.


12/04 20:33:02 - mmengine - INFO -
12/04 20:33:02 - mmengine - INFO - **********Environmental information**********
12/04 20:33:05 - mmengine - INFO - sys.platform: win32
12/04 20:33:05 - mmengine - INFO - Python: 3.8.20 (default, Oct  3 2024, 15:19:54) [MSC v.1929 64 bit (AMD64)]
12/04 20:33:05 - mmengine - INFO - CUDA available: True
12/04 20:33:05 - mmengine - INFO - MUSA available: False
12/04 20:33:05 - mmengine - INFO - numpy_random_seed: 2147483648
12/04 20:33:05 - mmengine - INFO - GPU 0: NVIDIA GeForce RTX 3080
12/04 20:33:05 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
12/04 20:33:05 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.8, V11.8.89
12/04 20:33:05 - mmengine - INFO - MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.29.30153 版
12/04 20:33:05 - mmengine - INFO - GCC: n/a
12/04 20:33:05 - mmengine - INFO - PyTorch: 2.0.0+cu118
12/04 20:33:05 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - C++ Version: 199711
  - MSVC 193431937
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)
  - OpenMP 2019
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX2
  - CUDA Runtime 11.8
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.7
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=C:/actions-runner/_work/pytorch/pytorch/builder/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj /FS -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.0.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

12/04 20:33:05 - mmengine - INFO - TorchVision: 0.15.1+cu118
12/04 20:33:05 - mmengine - INFO - OpenCV: 4.10.0
12/04 20:33:05 - mmengine - INFO - MMEngine: 0.10.5
12/04 20:33:05 - mmengine - INFO - MMCV: 2.1.0
12/04 20:33:05 - mmengine - INFO - MMCV Compiler: MSVC 192930148
12/04 20:33:05 - mmengine - INFO - MMCV CUDA Compiler: 11.8
12/04 20:33:05 - mmengine - INFO - MMDeploy: 1.3.1+3f8604b
12/04 20:33:05 - mmengine - INFO -

12/04 20:33:05 - mmengine - INFO - **********Backend information**********
12/04 20:33:05 - mmengine - INFO - tensorrt:    8.6.1
12/04 20:33:05 - mmengine - INFO - tensorrt custom ops: Available
12/04 20:33:06 - mmengine - INFO - ONNXRuntime: None
12/04 20:33:06 - mmengine - INFO - ONNXRuntime-gpu:     1.18.0
12/04 20:33:06 - mmengine - INFO - ONNXRuntime custom ops:      Available
12/04 20:33:06 - mmengine - INFO - pplnn:       None
12/04 20:33:06 - mmengine - INFO - ncnn:        None
12/04 20:33:06 - mmengine - INFO - snpe:        None
12/04 20:33:06 - mmengine - INFO - openvino:    None
12/04 20:33:06 - mmengine - INFO - torchscript: 2.0.0+cu118
12/04 20:33:06 - mmengine - INFO - torchscript custom ops:      NotAvailable
12/04 20:33:06 - mmengine - INFO - rknn-toolkit:        None
12/04 20:33:06 - mmengine - INFO - rknn-toolkit2:       None
12/04 20:33:06 - mmengine - INFO - ascend:      None
12/04 20:33:06 - mmengine - INFO - coreml:      None
12/04 20:33:06 - mmengine - INFO - tvm: None
12/04 20:33:06 - mmengine - INFO - vacc:        None
12/04 20:33:06 - mmengine - INFO -

12/04 20:33:06 - mmengine - INFO - **********Codebase information**********
12/04 20:33:06 - mmengine - INFO - mmdet:       3.3.0
12/04 20:33:06 - mmengine - INFO - mmseg:       None
12/04 20:33:06 - mmengine - INFO - mmpretrain:  1.2.0
12/04 20:33:06 - mmengine - INFO - mmocr:       None
12/04 20:33:06 - mmengine - INFO - mmagic:      None
12/04 20:33:06 - mmengine - INFO - mmdet3d:     None
12/04 20:33:06 - mmengine - INFO - mmpose:      1.3.2
12/04 20:33:06 - mmengine - INFO - mmrotate:    None
12/04 20:33:06 - mmengine - INFO - mmaction:    None
12/04 20:33:06 - mmengine - INFO - mmrazor:     None
12/04 20:33:06 - mmengine - INFO - mmyolo:      None

chemingmei avatar Dec 04 '24 12:12 chemingmei

Of course, I hope it can help you.

12/04 20:33:02 - mmengine - INFO -
12/04 20:33:02 - mmengine - INFO - **********Environmental information**********
12/04 20:33:05 - mmengine - INFO - sys.platform: win32
12/04 20:33:05 - mmengine - INFO - Python: 3.8.20 (default, Oct  3 2024, 15:19:54) [MSC v.1929 64 bit (AMD64)]
12/04 20:33:05 - mmengine - INFO - CUDA available: True
12/04 20:33:05 - mmengine - INFO - MUSA available: False
12/04 20:33:05 - mmengine - INFO - numpy_random_seed: 2147483648
12/04 20:33:05 - mmengine - INFO - GPU 0: NVIDIA GeForce RTX 3080
12/04 20:33:05 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8
12/04 20:33:05 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.8, V11.8.89
12/04 20:33:05 - mmengine - INFO - MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.29.30153 版
12/04 20:33:05 - mmengine - INFO - GCC: n/a
12/04 20:33:05 - mmengine - INFO - PyTorch: 2.0.0+cu118
12/04 20:33:05 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - C++ Version: 199711
  - MSVC 193431937
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)
  - OpenMP 2019
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX2
  - CUDA Runtime 11.8
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.7
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=C:/actions-runner/_work/pytorch/pytorch/builder/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj /FS -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.0.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

12/04 20:33:05 - mmengine - INFO - TorchVision: 0.15.1+cu118
12/04 20:33:05 - mmengine - INFO - OpenCV: 4.10.0
12/04 20:33:05 - mmengine - INFO - MMEngine: 0.10.5
12/04 20:33:05 - mmengine - INFO - MMCV: 2.1.0
12/04 20:33:05 - mmengine - INFO - MMCV Compiler: MSVC 192930148
12/04 20:33:05 - mmengine - INFO - MMCV CUDA Compiler: 11.8
12/04 20:33:05 - mmengine - INFO - MMDeploy: 1.3.1+3f8604b
12/04 20:33:05 - mmengine - INFO -

12/04 20:33:05 - mmengine - INFO - **********Backend information**********
12/04 20:33:05 - mmengine - INFO - tensorrt:    8.6.1
12/04 20:33:05 - mmengine - INFO - tensorrt custom ops: Available
12/04 20:33:06 - mmengine - INFO - ONNXRuntime: None
12/04 20:33:06 - mmengine - INFO - ONNXRuntime-gpu:     1.18.0
12/04 20:33:06 - mmengine - INFO - ONNXRuntime custom ops:      Available
12/04 20:33:06 - mmengine - INFO - pplnn:       None
12/04 20:33:06 - mmengine - INFO - ncnn:        None
12/04 20:33:06 - mmengine - INFO - snpe:        None
12/04 20:33:06 - mmengine - INFO - openvino:    None
12/04 20:33:06 - mmengine - INFO - torchscript: 2.0.0+cu118
12/04 20:33:06 - mmengine - INFO - torchscript custom ops:      NotAvailable
12/04 20:33:06 - mmengine - INFO - rknn-toolkit:        None
12/04 20:33:06 - mmengine - INFO - rknn-toolkit2:       None
12/04 20:33:06 - mmengine - INFO - ascend:      None
12/04 20:33:06 - mmengine - INFO - coreml:      None
12/04 20:33:06 - mmengine - INFO - tvm: None
12/04 20:33:06 - mmengine - INFO - vacc:        None
12/04 20:33:06 - mmengine - INFO -

12/04 20:33:06 - mmengine - INFO - **********Codebase information**********
12/04 20:33:06 - mmengine - INFO - mmdet:       3.3.0
12/04 20:33:06 - mmengine - INFO - mmseg:       None
12/04 20:33:06 - mmengine - INFO - mmpretrain:  1.2.0
12/04 20:33:06 - mmengine - INFO - mmocr:       None
12/04 20:33:06 - mmengine - INFO - mmagic:      None
12/04 20:33:06 - mmengine - INFO - mmdet3d:     None
12/04 20:33:06 - mmengine - INFO - mmpose:      1.3.2
12/04 20:33:06 - mmengine - INFO - mmrotate:    None
12/04 20:33:06 - mmengine - INFO - mmaction:    None
12/04 20:33:06 - mmengine - INFO - mmrazor:     None
12/04 20:33:06 - mmengine - INFO - mmyolo:      None

Thank you! I see that there is no onnxruntime and only onnxruntime-gpu. I will try this as well.

SushmaDG avatar Dec 04 '24 14:12 SushmaDG