mmdeploy
mmdeploy copied to clipboard
Published
20 hours ago •
open-mmlab
open-mmlab
[Bug] Cannot run ONNX converted MMOCR model (SAR)
Checklist
- [X] I have searched related issues but cannot get the expected help.
- [ ] 2. I have read the FAQ documentation but cannot get the expected help.
- [ ] 3. The bug has not been fixed in the latest version.
Describe the bug
I converted the SAR model to the ONNX format. I'm running a simple inference using Python SDK and getting an error: Segmentation fault.
Reproduction
mim download mmocr --config sar_resnet31_parallel-decoder_5e_st-sub_mj-sub_sa_real --dest .
python tools/deploy.py configs/mmocr/text-recognition/text-recognition_onnxruntime_dynamic.py sar_resnet31_parallel-decoder_5e_st-sub_mj-sub_sa_real.py sar_resnet31_parallel-deco
der_5e_st-sub_mj-sub_sa_real_20220915_171910-04eb4e75.pth demo/resources/text_recog.jpg --work-dir mmdeploy
_models/mmocr/sar/ort --device cpu --show --dump-info
python:
import cv2
# ================= init model ====================
text_recogizer = TextRecognizer(path='mmdeploy/mmdeploy_models/mmocr/sar/ort')
# ================= load image source ====================
# Load an image with opencv
img = cv2.imread('data/test2.jpg')
# ================= usage ====================
# With an image which represents only one word.
output = text_recogizer(img)
data/test2.jpg:
Environment
01/25 00:08:11 - mmengine - INFO -
01/25 00:08:11 - mmengine - INFO - **********Environmental information**********
01/25 00:08:11 - mmengine - INFO - sys.platform: linux
01/25 00:08:11 - mmengine - INFO - Python: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0]
01/25 00:08:11 - mmengine - INFO - CUDA available: False
01/25 00:08:11 - mmengine - INFO - MUSA available: False
01/25 00:08:11 - mmengine - INFO - numpy_random_seed: 2147483648
01/25 00:08:11 - mmengine - INFO - GCC: gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
01/25 00:08:11 - mmengine - INFO - PyTorch: 1.13.0+cpu
01/25 00:08:11 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
- GCC 9.3
- C++ Version: 201402
- Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.0, USE_CUDA=0, USE_CUDNN=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,
01/25 00:08:11 - mmengine - INFO - TorchVision: 0.14.0+cpu
01/25 00:08:11 - mmengine - INFO - OpenCV: 4.9.0
01/25 00:08:11 - mmengine - INFO - MMEngine: 0.10.3
01/25 00:08:11 - mmengine - INFO - MMCV: 2.0.0rc4
01/25 00:08:11 - mmengine - INFO - MMCV Compiler: GCC 7.3
01/25 00:08:11 - mmengine - INFO - MMCV CUDA Compiler: not available
01/25 00:08:11 - mmengine - INFO - MMDeploy: 1.3.1+bc75c9d
01/25 00:08:11 - mmengine - INFO -
01/25 00:08:11 - mmengine - INFO - **********Backend information**********
01/25 00:08:11 - mmengine - INFO - tensorrt: None
01/25 00:08:11 - mmengine - INFO - ONNXRuntime: 1.16.3
01/25 00:08:11 - mmengine - INFO - ONNXRuntime-gpu: None
01/25 00:08:11 - mmengine - INFO - ONNXRuntime custom ops: Available
01/25 00:08:11 - mmengine - INFO - pplnn: None
01/25 00:08:11 - mmengine - INFO - ncnn: None
01/25 00:08:11 - mmengine - INFO - snpe: None
01/25 00:08:12 - mmengine - INFO - openvino: None
01/25 00:08:12 - mmengine - INFO - torchscript: 1.13.0+cpu
01/25 00:08:12 - mmengine - INFO - torchscript custom ops: NotAvailable
01/25 00:08:12 - mmengine - INFO - rknn-toolkit: None
01/25 00:08:12 - mmengine - INFO - rknn-toolkit2: None
01/25 00:08:12 - mmengine - INFO - ascend: None
01/25 00:08:12 - mmengine - INFO - coreml: None
01/25 00:08:12 - mmengine - INFO - tvm: None
01/25 00:08:12 - mmengine - INFO - vacc: None
01/25 00:08:12 - mmengine - INFO -
01/25 00:08:12 - mmengine - INFO - **********Codebase information**********
01/25 00:08:12 - mmengine - INFO - mmdet: 3.1.0
01/25 00:08:12 - mmengine - INFO - mmseg: None
01/25 00:08:12 - mmengine - INFO - mmpretrain: None
01/25 00:08:12 - mmengine - INFO - mmocr: 1.0.1
01/25 00:08:12 - mmengine - INFO - mmagic: None
01/25 00:08:12 - mmengine - INFO - mmdet3d: None
01/25 00:08:12 - mmengine - INFO - mmpose: None
01/25 00:08:12 - mmengine - INFO - mmrotate: None
01/25 00:08:12 - mmengine - INFO - mmaction: None
01/25 00:08:12 - mmengine - INFO - mmrazor: None
01/25 00:08:12 - mmengine - INFO - mmyolo: None
Error traceback
loading libmmdeploy_ort_net.so ...
[2024-01-25 00:03:28.096] [mmdeploy] [info] [model.cpp:35] [DirectoryModel] Load model: "/home/kllis/mmdeploy/mmdeploy_models/mmocr/sar/ort"
[2024-01-25 00:03:28.097] [mmdeploy] [error] [compose.cpp:37] Unable to find Transform creator: PadToWidth. Available transforms: [("CenterCrop", 0), ("Collect", 0), ("Compose", 0), ("DefaultFormatBundle", 0), ("FormatShape", 0), ("ImageToTensor", 0), ("LetterResize", 0), ("Lift", 0), ("LoadImageFromFile", 0), ("Normalize", 0), ("Pad", 0), ("RescaleToHeight", 0), ("Resize", 0), ("ResizeOCR", 0), ("ShortScaleAspectJitter", 0), ("TenCrop", 0), ("ThreeCrop", 0), ("TopDownAffine", 0), ("TopDownGetBboxCenterScale", 0)]
[2024-01-25 00:03:28.097] [mmdeploy] [error] [task.cpp:99] error parsing config: {
"context": {
"device": "<any>",
"model": "<any>",
"stream": "<any>"
},
"input": [
"img"
],
"module": "Transform",
"name": "Preprocess",
"output": [
"prep_output"
],
"transforms": [
{
"type": "LoadImageFromFile"
},
{
"height": 48,
"max_width": 160,
"min_width": 48,
"type": "RescaleToHeight",
"width_divisor": 4
},
{
"type": "PadToWidth",
"width": 160
},
{
"mean": [
127,
127,
127
],
"std": [
127,
127,
127
],
"to_rgb": false,
"type": "Normalize"
},
{
"size_divisor": 1,
"type": "Pad"
},
{
"type": "DefaultFormatBundle"
},
{
"keys": [
"img"
],
"meta_keys": [
"pad_shape",
"img_shape",
"img_norm_cfg",
"flip_direction",
"ori_filename",
"ori_shape",
"valid_ratio",
"scale_factor",
"filename",
"flip",
"img_path"
],
"type": "Collect"
}
],
"type": "Task"
}
[2024-01-25 00:03:29.027] [mmdeploy] [info] [inference.cpp:54] ["img"] <- ["patches"]
[2024-01-25 00:03:29.027] [mmdeploy] [info] [inference.cpp:65] ["post_output"] -> ["texts"]
Segmentation fault
Hi @decadance-dance, I also have the same problem with master model. Have you fixed this bug yet? Can you give me a way to fix it? Thanks
I have the same issue. The reason of this issue is that PadToWidth is the specific transform process in mmocr but mmdeploy do not support it. So we can use resize before this issue solved.
