[WIP] New pipe

Open horcham opened this issue 1 year ago • 0 comments

Thank you for your contribution to the MindOCR repo. Before submitting this PR, please make sure:

[x] You have read the Contributing Guidelines on pull requests
[x] Your code builds clean without any errors or warnings
[x] You are using approved terminology
[x] You have added unit tests

Motivation

[x] 在线推理流水线升级，串行 -> 拓扑
[x] 在线推理流水线，命令

python pipeline/infer.py \
   --input_images_dir=deploy/py_infer/example/dataset/layout \
   --layout_algorithm=YOLOV8 \
   --layout_model_name_or_config=configs/layout/yolov8/yolov8n.yaml \
   --det_algorithm=DB \
   --det_model_name_or_config=configs/det/dbnet/db_r50_icdar15.yaml \
   --rec_algorithm=CRNN \
   --rec_model_name_or_config=configs/rec/crnn/crnn_resnet34.yaml \
   --cls_algorithm=MV3 \
   --cls_model_name_or_config=configs/cls/mobilenetv3/cls_mv3.yaml \
   --res_save_dir=res \
   --show_log=False \
   --vis_pipeline_save_dir=res \
   --visual_pipeline=False

支持模式： DET, REC, CLS, DET_REC, DET_CLS_REC, LAYOUT, LAYOUT_DET_REC, LAYOUT_DET_CLS_REC

[x] 支持离线、在线混推，使用方法
- 在yaml文件中定义predict
- 需新增字段
```
predict:
  # backend: MindSpore
  backend: MindSporeLite
  deive_target: Ascend
  device_id: 0
  max_device_memory: 8GB
  amp_level: O0
  mode: 0
  # ckpt_load_path: /root/.mindspore/models/dbnet_resnet50-c3a4aa24.ckpt
  ckpt_load_path: /data3/psw/zhanghongquan/experiment/new_pipe/mindocr/tools/output_folder/convert_folder/dbnet_resnet50_lite_dynamic.mindir
  ...
```
- 在yaml的predict中配置以下字段
  - backend: 可选MindSpore, MindSporeLite
  - deive_target: 可选Ascend, CPU，GPU. CPU，GPU未测试
  - device_id：该模型运行的设备
  - max_device_memory：MindSpore模式下最大显存
  - amp_level：优化选项
  - ckpt_load_path：MindSpore模式下为ckpt，MindSporeLite模式下为mindir
[x] 流水线可视化，可实时分析性能瓶颈

 0. Node:HandoutNode0, Has Processed:801, input queue:32     : 100%|███████████████████████████|32/32
 1. Node:DecodeNode0, Has Processed:784, input queue:16      : 100%|███████████████████████████|16/16
 2. Node:DetPreNode0, Has Processed:767, input queue:16      : 100%|███████████████████████████|16/16
 3. Node:DetInferNode0, Has Processed:750, input queue:15    :  94%|█████████████████████████▎ |15/16
 4. Node:DetPostNode0, Has Processed:734, input queue:16     : 100%|███████████████████████████|16/16
 5. Node:ClsPreNode0, Has Processed:717, input queue:16      : 100%|███████████████████████████|16/16
 6. Node:ClsInferNode0, Has Processed:700, input queue:16    : 100%|███████████████████████████|16/16
 7. Node:ClsPostNode0, Has Processed:683, input queue:16     : 100%|███████████████████████████|16/16
 8. Node:RecPreNode0, Has Processed:666, input queue:16      : 100%|███████████████████████████|16/16
 9. Node:RecInferNode0, Has Processed:649, input queue:16    : 100%|███████████████████████████|16/16
 10. Node:RecPostNode0, Has Processed:632, input queue:16    : 100%|███████████████████████████|16/16
 11. Node:CollectNode0, Has Processed:615, input queue:16    : 100%|███████████████████████████|16/16

[x] 线性流水线升级为拓扑流水线
[x] 加入Layout节点
[ ] 加入TableMaster节点

[ ] 功能测试

[ ] 全离线模型

[x] layout：yolov8

[x] det：dbnet_resnet60

pipeline/infer.py --input_images_dir=deploy/py_infer/example/dataset/layout --det_model_name_or_config=configs/det/dbnet/db_r50_icdar15.yaml --det_model_name_or_config=deploy/py_infer/src/configs/det/ppocr/ch_PP-OCRv4_det_cml.yaml --res_save_dir=res --show_log=False

[x] rec：crnn_resnet34

pipeline/infer.py --input_images_dir=deploy/py_infer/example/dataset/layout --rec_algorithm=CRNN --rec_model_name_or_config=configs/rec/crnn/crnn_resnet34.yaml --res_save_dir=res --show_log=False

[x] det_rec_cls

pipeline/infer.py --input_images_dir=deploy/py_infer/example/dataset/layout --det_algorithm=DB --det_model_name_or_config=configs/det/dbnet/db_r50_icdar15.yaml --rec_algorithm=CRNN --rec_model_name_or_config=configs/rec/crnn/crnn_resnet34.yaml --cls_algorithm=MV3 --cls_model_name_or_config=configs/cls/mobilenetv3/cls_mv3.yaml --res_save_dir=res --show_log=False --vis_pipeline_save_dir=res --visual_pipeline=False

[x] layout_det_rec_cls

python /pipeline/infer.py --input_images_dir=deploy/py_infer/example/dataset/layout --layout_algorithm=YOLOV8 --layout_model_name_or_config=configs/layout/yolov8/yolov8n.yaml --det_algorithm=DB --det_model_name_or_config=configs/det/dbnet/db_r50_icdar15.yaml --rec_algorithm=CRNN --rec_model_name_or_config=configs/rec/crnn/crnn_resnet34.yaml --cls_algorithm=MV3 --cls_model_name_or_config=configs/cls/mobilenetv3/cls_mv3.yaml --res_save_dir=res --show_log=False --vis_pipeline_save_dir=res --visual_pipeline=False

demo结果

[ ] 全在线模型
- [x] det_rec_cls
- [x] layout_det_rec_cls
[ ] 离线、在线模型
- [x] det_rec_cls，离线_在线_离线
- [x] layout_det_rec_cls，离线_在线_离线_在线

(Write your motivation for proposed changes here.)

Test Plan

(How should this PR be tested? Do you require special setup to run the test or repro the fixed bug?)

Related Issues and PRs

(Is this PR part of a group of changes? Link the other relevant PRs and Issues here. Use https://help.github.com/en/articles/closing-issues-using-keywords for help on GitHub syntax)

Apr 23 '24 01:04 horcham