Inferece SATRN with TensorRT

[Rm1n90]

Hi, I have converted sucessfully the SATRN model for text recogntion to ONNX and TensorRT without any issues but I have some problems during the inference. I have compiled all the basecodes from source: MMDeploy, MMocr, MMCV.

Im getting ´ModuleNotFoundError: No module named 'mmdeploy_python´ When Im trying to do the inference with the ocr.py which is provided in the demo/python. The main issue here is the file cannot import mmdeploy_python and I couldnt understand how to fix it since there is not such a file exist. How can i solve the import issue?

also I tried the inference_model() to validate the model and its working fine but have some questions:

This is the code

from mmdeploy.apis import inference_model

result = inference_model(
  model_cfg='/home/Documents/Dev/mmocr/configs/textrecog/satrn/satrn_small.py',
  deploy_cfg='/home/Documents/Dev/mmdeploy/configs/mmocr/text-recognition/text-recognition_tensorrt-int8_dynamic-32x32-32x640.py',
  backend_files=['/home/Documents/Dev/mmdeploy/work_dir/satrn/end2end.engine'],
  img='/home/Desktop/AdobeStock_231152703.jpg',
  device='cuda:0')
print(result)

My first question is about the output of the model. Here is the output

[{'text': '20187gz', 'score': [0.01894008181989193, 0.02894916944205761, 0.028949281200766563, 0.028977029025554657, 0.028331279754638672, 0.028976891189813614, 0.02900480106472969]}] As I understand the 'score' contains the score of each charachters of in the input image but why the scores are so low even the recognition of the characters are correct? Also, the 5th character is Z which SATRN recognized it as 7 wrongly but the confidence is simillar to other ones. How can I deal with scores to have proper range? Do I need to do any postprocess on the scores or is there any way to get the score of the word instead of character level scores ?

Thanks!

[tpoisonooo]

@lvhan028

[lvhan028]

Hi, @Rm1n90 For your first question about ´ModuleNotFoundError: No module named 'mmdeploy_python´, this is mmdeploy SDK python lib which is supposed to be built by enabling MMDEPLOY_BUILD_SDK_PYTHON_API.

Here is the build script described in https://mmdeploy.readthedocs.io/en/latest/01-how-to-build/linux-x86_64.html

• cuda + TensorRT

  cd ${MMDEPLOY_DIR}
  mkdir -p build && cd build
  cmake .. \
      -DMMDEPLOY_BUILD_SDK=ON \
      -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
      -DMMDEPLOY_BUILD_EXAMPLES=ON \
      -DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
      -Dpplcv_DIR=${PPLCV_DIR}/cuda-build/install/lib/cmake/ppl \
      -DTENSORRT_DIR=${TENSORRT_DIR} \
      -DCUDNN_DIR=${CUDNN_DIR}
  
  make -j$(nproc) && make install
  

After build, mmdeploy_python.cpython-37m-x86_64-linux-gnu.so will be found in build/lib directory, if your python version is 3.7.

Then you can do export PYTHONPATH=${MMDEPLOY_DIR}/build/lib:$PYTHONPATH to let python find package mmdeploy_python.

[lvhan028]

Regarding the second question about the low confidence of each recognized character, @gaotongxiao do you have any insight?

[lvhan028]

@Rm1n90 could you post your test image here?

[gaotongxiao]

I'm not sure what's wrong here. All I can confirm is SATRN's decoder outputs normalized (softmaxed) score at each time step.

https://github.com/open-mmlab/mmocr/blob/0a1787d6bcbec25649441a1f26e928ea7158e4c4/mmocr/models/textrecog/decoders/nrtr_decoder.py#L168-L169-171

I also ran ocr.py in MMOCR,

python mmocr/utils/ocr.py demo/demo_text_recog.jpg --det None --recog SATRN_sm --print-result

, which yielded a promising score:

{'text': 'star', 'score': 0.9981015622615814}

Though the score here is the "word" score, which was the average of the character scores.

https://github.com/open-mmlab/mmocr/blob/e2883074d8d7c27e858c6ccbb6b77e7513a28d45/mmocr/utils/ocr.py#L722-L724

[Rm1n90]

Hi @lvhan028, Sorry for late reply. Yes I followed the installation documantaion and built all of the basecodes from source but couldnt import mmdeploy_python. Now it fixed with your guidance! Thanks

@Rm1n90 could you post your test image here? It was a ranodm image which I downloaded it from internet. I will try to find it and upload it here.

@gaotongxiao I dont have issue with pytorch version. It work perfectly but when I convert the pytorch model to tensorRT the above issue will appear. I havent modify any codes, just followed the instruction for converting the weight file to the engine and inference the engine file and I still get the character score instead of word. will try to do the process one more time, maybe the problem is from my side.

[lvhan028]

Hi, @Rm1n90. It might be an issue of mmdeploy. We will try to reproduce it if you kindly provide us with your test image.