mmdeploy icon indicating copy to clipboard operation
mmdeploy copied to clipboard

Published 20 hours ago • verified publisher icon open-mmlab

python object_detection.py inference .engine error

Open jiaqizhang123-stack opened this issue 2 years ago • 8 comments

Hello，when I run 'python object_detection.py /home/zhang/shui_fp16 /home/zhang/mmdeploy/0_0152.bmp --device-name cuda ' [2022-07-22 17:11:23.879] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel' [2022-07-22 17:11:23.913] [mmdeploy] [info] [model.cpp:38] DirectoryModel successfully load sdk model /home/zhang/shui_fp16 [2022-07-22 17:11:23.971] [mmdeploy] [info] [common.h:29] config: { "context": { "device": "", "stream": "" }, "input": [ "image" ], "name": "mmdetection", "output": [ "det" ], "params": { "model": "" }, "type": "Inference" } [2022-07-22 17:11:23.971] [mmdeploy] [info] [common.h:29] config: { "context": { "device": "", "model": "", "stream": "" }, "input": [ "img" ], "module": "Transform", "name": "Preprocess", "output": [ "prep_output" ], "transforms": [ { "type": "LoadImageFromFile" }, { "keep_ratio": false, "size": [ 480, 1280 ], "type": "Resize" }, { "mean": [ 123.675, 116.28, 103.53 ], "std": [ 58.395, 57.12, 57.375 ], "to_rgb": true, "type": "Normalize" }, { "size_divisor": 32, "type": "Pad" }, { "type": "DefaultFormatBundle" }, { "keys": [ "img" ], "meta_keys": [ "pad_shape", "flip_direction", "filename", "flip", "img_norm_cfg", "valid_ratio", "ori_shape", "scale_factor", "ori_filename", "img_shape" ], "type": "Collect" } ], "type": "Task" } [2022-07-22 17:11:23.971] [mmdeploy] [info] [common.h:29] config: { "context": { "device": "", "model": "", "stream": "" }, "input": [ "img" ], "module": "Transform", "name": "Preprocess", "output": [ "prep_output" ], "transforms": [ { "type": "LoadImageFromFile" }, { "keep_ratio": false, "size": [ 480, 1280 ], "type": "Resize" }, { "mean": [ 123.675, 116.28, 103.53 ], "std": [ 58.395, 57.12, 57.375 ], "to_rgb": true, "type": "Normalize" }, { "size_divisor": 32, "type": "Pad" }, { "type": "DefaultFormatBundle" }, { "keys": [ "img" ], "meta_keys": [ "pad_shape", "flip_direction", "filename", "flip", "img_norm_cfg", "valid_ratio", "ori_shape", "scale_factor", "ori_filename", "img_shape" ], "type": "Collect" } ], "type": "Task" } [2022-07-22 17:11:23.971] [mmdeploy] [info] [common.h:29] config: { "context": { "device": "", "model": "", "stream": "" }, "input": [ "prep_output" ], "input_map": { "img": "input" }, "module": "Net", "name": "maskrcnn", "output": [ "infer_output" ], "type": "Task" } [2022-07-22 17:11:23.971] [mmdeploy] [info] [common.h:29] config: { "context": { "device": "", "model": "", "stream": "" }, "input": [ "prep_output" ], "input_map": { "img": "input" }, "module": "Net", "name": "maskrcnn", "output": [ "infer_output" ], "type": "Task" } [2022-07-22 17:11:23.971] [mmdeploy] [info] [cuda_device.cpp:61] Default CUDA allocator initialized [2022-07-22 17:11:24.312] [mmdeploy] [warning] [trt_net.cpp:24] TRTNet: TensorRT was linked against cuBLAS/cuBLAS LT 10.2.3 but loaded cuBLAS/cuBLAS LT 10.2.2 [2022-07-22 17:11:24.441] [mmdeploy] [warning] [trt_net.cpp:24] TRTNet: TensorRT was linked against cuBLAS/cuBLAS LT 10.2.3 but loaded cuBLAS/cuBLAS LT 10.2.2 [2022-07-22 17:11:24.446] [mmdeploy] [error] [trt_net.cpp:28] TRTNet: 1: [graphContext.h::MyelinGraphContext::24] Error Code 1: Myelin (Compiled against cuBLAS 10.2.3.0 but running against cuBLAS 10.2.2.0.) [2022-07-22 17:11:24.450] [mmdeploy] [error] [trt_net.cpp:28] TRTNet: 1: [graphContext.h::MyelinGraphContext::24] Error Code 1: Myelin (Compiled against cuBLAS 10.2.3.0 but running against cuBLAS 10.2.2.0.) [2022-07-22 17:11:24.455] [mmdeploy] [info] [common.h:29] config: { "component": "ResizeInstanceMask", "context": { "device": "", "model": "", "stream": "" }, "input": [ "prep_output", "infer_output" ], "module": "mmdet", "name": "postprocess", "output": [ "post_output" ], "params": { "mask_thr_binary": 0.5, "min_bbox_size": 0, "rcnn": { "mask_thr_binary": 0.5, "max_per_img": 90, "nms": { "iou_threshold": 0.5, "type": "nms" }, "score_thr": 0.1 }, "rpn": { "max_per_img": 1000, "min_bbox_size": 0, "nms": { "iou_threshold": 0.7, "type": "nms" }, "nms_pre": 1000 }, "score_thr": 0.1 }, "type": "Task" } [2022-07-22 17:11:24.455] [mmdeploy] [info] [common.h:29] config: { "component": "ResizeInstanceMask", "context": { "device": "", "model": "", "stream": "" }, "input": [ "prep_output", "infer_output" ], "module": "mmdet", "name": "postprocess", "output": [ "post_output" ], "params": { "mask_thr_binary": 0.5, "min_bbox_size": 0, "rcnn": { "mask_thr_binary": 0.5, "max_per_img": 90, "nms": { "iou_threshold": 0.5, "type": "nms" }, "score_thr": 0.1 }, "rpn": { "max_per_img": 1000, "min_bbox_size": 0, "nms": { "iou_threshold": 0.7, "type": "nms" }, "nms_pre": 1000 }, "score_thr": 0.1 }, "type": "Task" } [2022-07-22 17:11:24.455] [mmdeploy] [warning] [bulk.h:39] fallback Bulk implementation [2022-07-22 17:11:24.456] [mmdeploy] [warning] [bulk.h:39] fallback Bulk implementation [2022-07-22 17:11:24.456] [mmdeploy] [error] [trt_net.cpp:28] TRTNet: 3: [executionContext.cpp::setBindingDimensions::926] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::setBindingDimensions::926, condition: mOptimizationProfile >= 0 && mOptimizationProfile < mEngine.getNbOptimizationProfiles() ) 已放弃 (核心已转储) The problem occurred, but the C++ SDK works.

jiaqizhang123-stack avatar Jul 22 '22 09:07 jiaqizhang123-stack

Can you print the result of python tools/check_env.py

irexyc avatar Jul 22 '22 09:07 irexyc

022-07-22 17:33:45,989 - mmdeploy - INFO -

2022-07-22 17:33:45,989 - mmdeploy - INFO - Environmental information fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this: 'git [...] -- [...]' 2022-07-22 17:33:47,081 - mmdeploy - INFO - sys.platform: linux 2022-07-22 17:33:47,081 - mmdeploy - INFO - Python: 3.9.12 (main, Jun 1 2022, 11:38:51) [GCC 7.5.0] 2022-07-22 17:33:47,082 - mmdeploy - INFO - CUDA available: True 2022-07-22 17:33:47,082 - mmdeploy - INFO - GPU 0: NVIDIA GeForce GTX 1050 Ti 2022-07-22 17:33:47,082 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda-10.2 2022-07-22 17:33:47,082 - mmdeploy - INFO - NVCC: Cuda compilation tools, release 10.2, V10.2.89 2022-07-22 17:33:47,082 - mmdeploy - INFO - GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 2022-07-22 17:33:47,082 - mmdeploy - INFO - PyTorch: 1.8.0 2022-07-22 17:33:47,082 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:

  • GCC 7.3
  • C++ Version: 201402
  • Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  • Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
  • OpenMP 201511 (a.k.a. OpenMP 4.5)
  • NNPACK is enabled
  • CPU capability usage: AVX2
  • CUDA Runtime 10.2
  • NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70
  • CuDNN 7.6.5
  • Magma 2.5.2
  • Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

2022-07-22 17:33:47,082 - mmdeploy - INFO - TorchVision: 0.9.0 2022-07-22 17:33:47,082 - mmdeploy - INFO - OpenCV: 4.5.3 2022-07-22 17:33:47,082 - mmdeploy - INFO - MMCV: 1.4.0 2022-07-22 17:33:47,082 - mmdeploy - INFO - MMCV Compiler: GCC 7.3 2022-07-22 17:33:47,082 - mmdeploy - INFO - MMCV CUDA Compiler: 10.2 2022-07-22 17:33:47,082 - mmdeploy - INFO - MMDeploy: 0.5.0+HEAD 2022-07-22 17:33:47,082 - mmdeploy - INFO -

2022-07-22 17:33:47,082 - mmdeploy - INFO - Backend information 2022-07-22 17:33:47,476 - mmdeploy - INFO - onnxruntime: 1.10.0 ops_is_avaliable : True 2022-07-22 17:33:47,497 - mmdeploy - INFO - tensorrt: 8.2.3.0 ops_is_avaliable : True 2022-07-22 17:33:47,515 - mmdeploy - INFO - ncnn: None ops_is_avaliable : False 2022-07-22 17:33:47,516 - mmdeploy - INFO - pplnn_is_avaliable: False 2022-07-22 17:33:47,517 - mmdeploy - INFO - openvino_is_avaliable: False 2022-07-22 17:33:47,517 - mmdeploy - INFO -

2022-07-22 17:33:47,517 - mmdeploy - INFO - Codebase information 2022-07-22 17:33:47,518 - mmdeploy - INFO - mmdet: 2.25.0 2022-07-22 17:33:47,518 - mmdeploy - INFO - mmseg: None 2022-07-22 17:33:47,518 - mmdeploy - INFO - mmcls: None 2022-07-22 17:33:47,518 - mmdeploy - INFO - mmocr: None 2022-07-22 17:33:47,518 - mmdeploy - INFO - mmedit: None 2022-07-22 17:33:47,518 - mmdeploy - INFO - mmdet3d: None 2022-07-22 17:33:47,518 - mmdeploy - INFO - mmpose: None 2022-07-22 17:33:47,518 - mmdeploy - INFO - mmrotate: None

jiaqizhang123-stack avatar Jul 22 '22 09:07 jiaqizhang123-stack

And when I use './object_detection cuda /media/zhang/DATA11/work_dir/TensorRT/trans_segmask3onnx/epoch_36 /home/zhang/mmdeploy/0_0152.bmp' inference .onnx. The problem occured.

[2022-07-22 17:31:45.346] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel' [2022-07-22 17:31:45.378] [mmdeploy] [info] [model.cpp:38] DirectoryModel successfully load sdk model /media/zhang/DATA11/work_dir/TensorRT/trans_segmask3onnx/epoch_36 [2022-07-22 17:31:45.429] [mmdeploy] [info] [common.h:29] config: { "context": { "device": "", "stream": "" }, "input": [ "image" ], "name": "mmdetection", "output": [ "det" ], "params": { "model": "" }, "type": "Inference" } [2022-07-22 17:31:45.429] [mmdeploy] [info] [common.h:29] config: { "context": { "device": "", "model": "", "stream": "" }, "input": [ "img" ], "module": "Transform", "name": "Preprocess", "output": [ "prep_output" ], "transforms": [ { "type": "LoadImageFromFile" }, { "keep_ratio": true, "size": [ 800, 1333 ], "type": "Resize" }, { "mean": [ 123.675, 116.28, 103.53 ], "std": [ 58.395, 57.12, 57.375 ], "to_rgb": true, "type": "Normalize" }, { "size_divisor": 32, "type": "Pad" }, { "type": "DefaultFormatBundle" }, { "keys": [ "img" ], "meta_keys": [ "img_norm_cfg", "flip", "ori_shape", "img_shape", "filename", "pad_shape", "flip_direction", "scale_factor", "valid_ratio", "ori_filename" ], "type": "Collect" } ], "type": "Task" } [2022-07-22 17:31:45.429] [mmdeploy] [info] [common.h:29] config: { "context": { "device": "", "model": "", "stream": "" }, "input": [ "img" ], "module": "Transform", "name": "Preprocess", "output": [ "prep_output" ], "transforms": [ { "type": "LoadImageFromFile" }, { "keep_ratio": true, "size": [ 800, 1333 ], "type": "Resize" }, { "mean": [ 123.675, 116.28, 103.53 ], "std": [ 58.395, 57.12, 57.375 ], "to_rgb": true, "type": "Normalize" }, { "size_divisor": 32, "type": "Pad" }, { "type": "DefaultFormatBundle" }, { "keys": [ "img" ], "meta_keys": [ "img_norm_cfg", "flip", "ori_shape", "img_shape", "filename", "pad_shape", "flip_direction", "scale_factor", "valid_ratio", "ori_filename" ], "type": "Collect" } ], "type": "Task" } [2022-07-22 17:31:45.429] [mmdeploy] [info] [common.h:29] config: { "context": { "device": "", "model": "", "stream": "" }, "input": [ "prep_output" ], "input_map": { "img": "input" }, "module": "Net", "name": "maskrcnn", "output": [ "infer_output" ], "type": "Task" } [2022-07-22 17:31:45.429] [mmdeploy] [info] [common.h:29] config: { "context": { "device": "", "model": "", "stream": "" }, "input": [ "prep_output" ], "input_map": { "img": "input" }, "module": "Net", "name": "maskrcnn", "output": [ "infer_output" ], "type": "Task" } [2022-07-22 17:31:45.429] [mmdeploy] [info] [cuda_device.cpp:61] Default CUDA allocator initialized [2022-07-22 17:31:45.429] [mmdeploy] [error] [net_module.cpp:37] Net backend not found: onnxruntime [2022-07-22 17:31:45.429] [mmdeploy] [error] [task.cpp:67] error parsing config: { "context": { "device": "", "model": "", "stream": "" }, "input": [ "prep_output" ], "input_map": { "img": "input" }, "module": "Net", "name": "maskrcnn", "output": [ "infer_output" ], "type": "Task" } [2022-07-22 17:31:45.429] [mmdeploy] [error] [pipeline.cpp:146] could not create maskrcnn: Task [2022-07-22 17:31:45.429] [mmdeploy] [error] [pipeline.cpp:160] error parsing config: unknown (6) @ /mmdeploy/csrc/graph/pipeline.cpp:147 [2022-07-22 17:31:45.429] [mmdeploy] [error] [pipeline.cpp:27] exception caught: unknown (6) @ /mmdeploy/csrc/graph/pipeline.cpp:161 failed to create detector, code: 6

jiaqizhang123-stack avatar Jul 22 '22 09:07 jiaqizhang123-stack

We will check the python inference trt problem.

For the onnxruntime error, it seems that you didn't build mmdeploy with onnxruntime. If you want to build mmdeploy with trt and ort support, you shoud pass -DMMDEPLOY_TARGET_BACKENDS="trt;ort"

irexyc avatar Jul 22 '22 11:07 irexyc

Thanks for your answer, but I am using the precompiled package of mmdeploy, and "2022-07-22 17:33:47,476 - mmdeploy - INFO - onnxruntime: 1.10.0 ops_is_avaliable : True" is ok. Do I need to recompile it myself?

jiaqizhang123-stack avatar Jul 22 '22 11:07 jiaqizhang123-stack

I don't know how to reproduce your problem. Because If you use the precompiled package, python api(if you run python object_detection.py) and c api should load same libraries so it shouldn't happen that python api goes wrong while c api runs success.

Please don't mix use the precompiled package and your build version. You may supply a clearer reproduction path.

irexyc avatar Jul 25 '22 06:07 irexyc

For python, I don't know how to run object_detection.py. When I put it in this directory it will not give an error. But above problems occured. 图片 For c++, by“mkdir -p build && cd build cmake -DMMDeploy_DIR=${MMDEPLOY_DIR}/build/install/lib/cmake/MMDeploy .. make object_detection”, the object_detection was ok.

jiaqizhang123-stack avatar Jul 25 '22 07:07 jiaqizhang123-stack

From the above information, I don't konw how to reproduce your problem.

Please describe how to reproduce the python problem in detail or you can add our wechat group and ask. https://github.com/open-mmlab/mmdeploy/blob/master/README_zh-CN.md

irexyc avatar Jul 25 '22 12:07 irexyc