mmdeploy icon indicating copy to clipboard operation
mmdeploy copied to clipboard
verified publisher icon open-mmlab

Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)

Open JackIRose opened this issue 1 year ago • 0 comments

Checklist

  • [x] I have searched related issues but cannot get the expected help.
  • [X] 2. I have read the FAQ documentation but cannot get the expected help.
  • [X] 3. The bug has not been fixed in the latest version.

Describe the bug

使用python tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-64x64-800x800.py rtmdet_tiny_8xb32-300e_coco.py rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth det.jpg --work-dir mmdeploy/RTMdet_t_trt --device cuda --dump-info后，能够成功生成engine文件并输出测试图片。

但是当我使用C++加载engine文件进行推理时会报一下错误： [03/05/2024-13:55:15] [TRT] [E] 1: [pluginV2Runner.cpp::nvinfer1::rt::load::293] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry) [03/05/2024-13:55:15] [TRT] [E] 4: [runtime.cpp::nvinfer1::Runtime::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.)

Reproduction

int main(int argc, char** argv) { // create a model using the API directly and serialize it to a stream char* trtModelStream{ nullptr }; size_t size{ 0 }; std::cout << "assssssssss" << std::endl;

std::ifstream file("E:/工厂机器视觉-行为动作识别/mmdeploy-main/RTM-ins/end2end.engine", std::ios::binary);
if (file.good()) {
    file.seekg(0, file.end);
    size = file.tellg();
    file.seekg(0, file.beg);
    trtModelStream = new char[size];
    assert(trtModelStream);
    file.read(trtModelStream, size);
    file.close();
}


Logger m_logger;
IRuntime* runtime = createInferRuntime(m_logger);
assert(runtime != nullptr);
std::cout << "1111111111111111111" << std::endl;


// 初始化插件库
bool success = initLibNvInferPlugins(&m_logger, "");
std::cout << "success: " << success << std::endl;


ICudaEngine* engine = runtime->deserializeCudaEngine(trtModelStream, size, nullptr);
assert(engine != nullptr);

std::cout << "2222222222222222222" << std::endl;

IExecutionContext* context = engine->createExecutionContext();
assert(context != nullptr);
std::cout << "3333333333333333333" << std::endl;

// generate input data
float data[BATCH_SIZE * 3 * IN_H * IN_W];
for (int i = 0; i < BATCH_SIZE * 3 * IN_H * IN_W; i++)
    data[i] = 1;

//测试doInferece函数100次，计算平均时间

auto start = std::chrono::high_resolution_clock::now();
// Run inference
float prob[BATCH_SIZE * 3 * IN_H * IN_W / 4];
for (int i = 0; i < 100; i++) {
    doInference(*context, data, prob, BATCH_SIZE);
}
std::cout<<prob[0]<<std::endl;  
auto end = std::chrono::high_resolution_clock::now();
std::cout << "RTMpose处理一帧图片所需时间" << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count()/100 << "ms" << std::endl;


// Destroy the engine
context->destroy();
engine->destroy();
runtime->destroy();
return 0;

}

Environment

03/05 15:58:48 - mmengine - INFO - 

03/05 15:58:48 - mmengine - INFO - **********Environmental information**********
03/05 15:59:02 - mmengine - INFO - sys.platform: win32
03/05 15:59:02 - mmengine - INFO - Python: 3.8.18 (default, Sep 11 2023, 13:39:12) [MSC v.1916 64 bit (AMD64)]
03/05 15:59:02 - mmengine - INFO - CUDA available: True
03/05 15:59:02 - mmengine - INFO - MUSA available: False
03/05 15:59:02 - mmengine - INFO - numpy_random_seed: 2147483648
03/05 15:59:02 - mmengine - INFO - GPU 0: Quadro P2200
03/05 15:59:02 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5
03/05 15:59:02 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.5, V11.5.50
03/05 15:59:02 - mmengine - INFO - MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.29.30154 版
03/05 15:59:02 - mmengine - INFO - GCC: n/a
03/05 15:59:02 - mmengine - INFO - PyTorch: 1.12.0
03/05 15:59:02 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - C++ Version: 199711
  - MSVC 192829337
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 2019
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.3.2  (built against CUDA 11.5)
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -IC:/cb/pytorch_1000000000000/work/mkl/include -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

03/05 15:59:02 - mmengine - INFO - TorchVision: 0.13.0
03/05 15:59:02 - mmengine - INFO - OpenCV: 4.9.0
03/05 15:59:02 - mmengine - INFO - MMEngine: 0.10.3
03/05 15:59:02 - mmengine - INFO - MMCV: 2.1.0
03/05 15:59:02 - mmengine - INFO - MMCV Compiler: MSVC 192930148
03/05 15:59:02 - mmengine - INFO - MMCV CUDA Compiler: 11.3
03/05 15:59:02 - mmengine - INFO - MMDeploy: 1.3.1+unknown
03/05 15:59:02 - mmengine - INFO -

03/05 15:59:02 - mmengine - INFO - **********Backend information**********
03/05 15:59:03 - mmengine - INFO - tensorrt:    8.4.1.5
03/05 15:59:03 - mmengine - INFO - ONNXRuntime: None
03/05 15:59:03 - mmengine - INFO - ONNXRuntime-gpu:     1.17.1
03/05 15:59:03 - mmengine - INFO - ONNXRuntime custom ops:      Available
03/05 15:59:03 - mmengine - INFO - pplnn:       None
03/05 15:59:03 - mmengine - INFO - ncnn:        None
03/05 15:59:03 - mmengine - INFO - snpe:        None
03/05 15:59:03 - mmengine - INFO - openvino:    None
03/05 15:59:03 - mmengine - INFO - torchscript: 1.12.0
03/05 15:59:03 - mmengine - INFO - torchscript custom ops:      NotAvailable
03/05 15:59:03 - mmengine - INFO - rknn-toolkit:        None
03/05 15:59:03 - mmengine - INFO - rknn-toolkit2:       None
03/05 15:59:03 - mmengine - INFO - ascend:      None
03/05 15:59:04 - mmengine - INFO - coreml:      None
03/05 15:59:04 - mmengine - INFO - tvm: None
03/05 15:59:04 - mmengine - INFO - vacc:        None
03/05 15:59:04 - mmengine - INFO -

03/05 15:59:04 - mmengine - INFO - **********Codebase information**********
03/05 15:59:04 - mmengine - INFO - mmdet:       3.3.0
03/05 15:59:04 - mmengine - INFO - mmseg:       None
03/05 15:59:04 - mmengine - INFO - mmpretrain:  None
03/05 15:59:04 - mmengine - INFO - mmocr:       None
03/05 15:59:04 - mmengine - INFO - mmagic:      None
03/05 15:59:04 - mmengine - INFO - mmdet3d:     None
03/05 15:59:04 - mmengine - INFO - mmpose:      1.3.1
03/05 15:59:04 - mmengine - INFO - mmrotate:    None
03/05 15:59:04 - mmengine - INFO - mmaction:    None
03/05 15:59:04 - mmengine - INFO - mmrazor:     None
03/05 15:59:04 - mmengine - INFO - mmyolo:      None

Error traceback

04/19/2024-17:31:40] [E] [TRT] 1: [pluginV2Runner.cpp::nvinfer1::rt::load::293] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry) [04/19/2024-17:31:40] [E] [TRT] 4: [runtime.cpp::nvinfer1::Runtime::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.

JackIRose avatar Mar 05 '24 08:03 JackIRose