0.8.0版本ocrv3的predict方法直接程序退出

[yz2yz]

0.8.0版本ocrv3的predict方法直接程序退出，0.7.0版本没有这个问题。测试脚本如下，使用是paddleocr的公开模型，48行执行异常退出，没有报错信息：

import os os.environ['CUDA_MODULE_LOADING'] = 'LAZY'

import fastdeploy as fd from fastdeploy import ModelFormat import cv2 import numpy as np import os import psutil

det_model_file = 'weights/ch_PP-OCRv3_det_infer/inference.pdmodel' det_params_file = 'weights/ch_PP-OCRv3_det_infer/inference.pdiparams' rec_model_file = 'weights/ch_PP-OCRv3_rec_infer/inference.pdmodel' rec_params_file = 'weights/ch_PP-OCRv3_rec_infer/inference.pdiparams'

option_det = fd.RuntimeOption() option_det.use_gpu() option_det.use_trt_backend() # TensorRT option_det.set_trt_cache_file('weights/ch_PP-OCRv3_det_infer/ch_PP-OCRv3_det.trt') option_det.set_trt_input_shape(tensor_name='x', min_shape=[1,3,224,224], opt_shape=[1,3,896,896], max_shape=[1,3,3584,3584]) option_det.set_trt_max_workspace_size(1 << 28) # 256M 1GB

det_model = fd.vision.ocr.DBDetector( model_file=det_model_file, params_file=det_params_file, runtime_option=option_det, model_format=ModelFormat.PADDLE) print('det_model:', det_model)

option_rec = fd.RuntimeOption() option_rec.use_gpu() option_rec.use_trt_backend() # TensorRT option_rec.set_trt_cache_file('weights/ch_PP-OCRv3_rec_infer/ch_PP-OCRv3_rec.trt') option_rec.set_trt_input_shape('x', (1,3,48,8), (1,3,48,320), (1,3,48,1792)) option_rec.set_trt_max_workspace_size(1 << 28) # 256M 1GB rec_model = fd.vision.ocr.Recognizer( model_file=rec_model_file, params_file=rec_params_file, label_path='weights/ch_PP-OCRv3_rec_infer/ppocr_keys_v1.txt', runtime_option=option_rec, model_format=ModelFormat.PADDLE) print('rec_model:', rec_model)

model = fd.vision.ocr.PPOCRv3(det_model=det_model, cls_model=None, rec_model=rec_model) print('model:', model)

fname = 'imgs/KN511816.jpg' img = cv2.imdecode(np.fromfile(fname, dtype=np.uint8), cv2.IMREAD_COLOR) result = model.predict(img) print(type(result)) print(result)

mem = psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024 / 1024 print(f'mem:{mem:.4f}GB')

[yz2yz]

使用最新版本0.0.0的FP32进行测试，不需要传入cls也能正常推理； 但是，开启FP16后，清理掉前面生成的trt文件，就报错：

[FastDeploy][INFO]: Successfully found CUDA ToolKit from system PATH env -> D:\cuda113\cuda\bin [INFO] fastdeploy/backends/tensorrt/trt_backend.cc(478)::fastdeploy::TrtBackend::BuildTrtEngine [TrtBackend] Use FP16 to inference. [INFO] fastdeploy/backends/tensorrt/trt_backend.cc(483)::fastdeploy::TrtBackend::BuildTrtEngine Start to building TensorRT Engine... [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(238)::fastdeploy::FDTrtLogger::log 2: [pluginV2DynamicExtRunner.cpp::nvinfer1::rt::cuda::PluginV2DynamicExtRunner::execute::115] Error Code 2: Internal Error (Assertion status == kSTATUS_SUCCESS failed. ) [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(238)::fastdeploy::FDTrtLogger::log 2: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::636] Error Code 2: Internal Error (Assertion engine != nullptr failed. ) [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(546)::fastdeploy::TrtBackend::BuildTrtEngine Failed to call buildSerializedNetwork(). [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(659)::fastdeploy::TrtBackend::CreateTrtEngineFromOnnx Failed to build tensorrt engine. [INFO] fastdeploy/runtime.cc(506)::fastdeploy::Runtime::Init Runtime initialized with Backend::TRT in Device::GPU. det_model: <fastdeploy.vision.ocr.ppocr.DBDetector object at 0x000002501ADE7BB0>

[INFO] fastdeploy/backends/tensorrt/trt_backend.cc(478)::fastdeploy::TrtBackend::BuildTrtEngine [TrtBackend] Use FP16 to inference. [INFO] fastdeploy/backends/tensorrt/trt_backend.cc(483)::fastdeploy::TrtBackend::BuildTrtEngine Start to building TensorRT Engine... [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(238)::fastdeploy::FDTrtLogger::log 2: [pluginV2DynamicExtRunner.cpp::nvinfer1::rt::cuda::PluginV2DynamicExtRunner::execute::115] Error Code 2: Internal Error (Assertion status == kSTATUS_SUCCESS failed. ) [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(238)::fastdeploy::FDTrtLogger::log 2: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::636] Error Code 2: Internal Error (Assertion engine != nullptr failed. ) [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(546)::fastdeploy::TrtBackend::BuildTrtEngine Failed to call buildSerializedNetwork(). [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(659)::fastdeploy::TrtBackend::CreateTrtEngineFromOnnx Failed to build tensorrt engine. [INFO] fastdeploy/runtime.cc(506)::fastdeploy::Runtime::Init Runtime initialized with Backend::TRT in Device::GPU. rec_model: <fastdeploy.vision.ocr.ppocr.Recognizer object at 0x000002503897F0D0> model: <fastdeploy.vision.ocr.ppocr.PPOCRv3 object at 0x000002503897F760> [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(386)::fastdeploy::TrtBackend::SetInputs TRTBackend SetInputs not find name:x

[yunyaoXYY]

使用最新版本0.0.0的FP32进行测试，不需要传入cls也能正常推理； 但是，开启FP16后，清理掉前面生成的trt文件，就报错：

[FastDeploy][INFO]: Successfully found CUDA ToolKit from system PATH env -> D:\cuda113\cuda\bin [INFO] fastdeploy/backends/tensorrt/trt_backend.cc(478)::fastdeploy::TrtBackend::BuildTrtEngine [TrtBackend] Use FP16 to inference. [INFO] fastdeploy/backends/tensorrt/trt_backend.cc(483)::fastdeploy::TrtBackend::BuildTrtEngine Start to building TensorRT Engine... [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(238)::fastdeploy::FDTrtLogger::log 2: [pluginV2DynamicExtRunner.cpp::nvinfer1::rt::cuda::PluginV2DynamicExtRunner::execute::115] Error Code 2: Internal Error (Assertion status == kSTATUS_SUCCESS failed. ) [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(238)::fastdeploy::FDTrtLogger::log 2: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::636] Error Code 2: Internal Error (Assertion engine != nullptr failed. ) [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(546)::fastdeploy::TrtBackend::BuildTrtEngine Failed to call buildSerializedNetwork(). [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(659)::fastdeploy::TrtBackend::CreateTrtEngineFromOnnx Failed to build tensorrt engine. [INFO] fastdeploy/runtime.cc(506)::fastdeploy::Runtime::Init Runtime initialized with Backend::TRT in Device::GPU. det_model: <fastdeploy.vision.ocr.ppocr.DBDetector object at 0x000002501ADE7BB0>

[INFO] fastdeploy/backends/tensorrt/trt_backend.cc(478)::fastdeploy::TrtBackend::BuildTrtEngine [TrtBackend] Use FP16 to inference. [INFO] fastdeploy/backends/tensorrt/trt_backend.cc(483)::fastdeploy::TrtBackend::BuildTrtEngine Start to building TensorRT Engine... [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(238)::fastdeploy::FDTrtLogger::log 2: [pluginV2DynamicExtRunner.cpp::nvinfer1::rt::cuda::PluginV2DynamicExtRunner::execute::115] Error Code 2: Internal Error (Assertion status == kSTATUS_SUCCESS failed. ) [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(238)::fastdeploy::FDTrtLogger::log 2: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::636] Error Code 2: Internal Error (Assertion engine != nullptr failed. ) [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(546)::fastdeploy::TrtBackend::BuildTrtEngine Failed to call buildSerializedNetwork(). [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(659)::fastdeploy::TrtBackend::CreateTrtEngineFromOnnx Failed to build tensorrt engine. [INFO] fastdeploy/runtime.cc(506)::fastdeploy::Runtime::Init Runtime initialized with Backend::TRT in Device::GPU. rec_model: <fastdeploy.vision.ocr.ppocr.Recognizer object at 0x000002503897F0D0> model: <fastdeploy.vision.ocr.ppocr.PPOCRv3 object at 0x000002503897F760> [ERROR] fastdeploy/backends/tensorrt/trt_backend.cc(386)::fastdeploy::TrtBackend::SetInputs TRTBackend SetInputs not find name:x

Hi 我这边在支持FP16计算的GPU上，用TensorRT8.4.1.5复现了没有这个问题呢

[jiangjiajun]

此ISSUE由于一年未更新，将会关闭处理，如有需要，可再次更新打开。