Iteration Plan July-Aug

[innerlee]

This month (mid May - mid June), we will continue to improve the usage of the repo. More documentation and more demo will be added. Plus, we will continue the work left in last month.

• [ ] Improve test coverage (> 90%) @all
• [x] Imporve doc on demo
• [ ] Add more PWC meta data
• [ ] Support bioes format
• [ ] Improve handling of vacab_size
• [ ] Improve general code quality
• [ ] Online demo @yuexy
• [ ] Add demo cfg @innerlee @cuhk-hbsun
• [ ] benchmarks with paddleocr, chineseocr, easyocr, chineseocr_lite
  • [ ] Algorithm coverage @jeffreykuang
  • [ ] Feature set @jeffreykuang
• [ ] Get rid of orientation operations in data preparation https://github.com/open-mmlab/mmocr/issues/112
• [ ] KIE Improvements
  • [x] Multiple gpu training in KIE @innerlee https://github.com/open-mmlab/mmocr/pull/234
  • [x] Profile & accelerate KIE training @innerlee
  • [x] Skip image pipeline when no visual feature needed @innerlee
  • [x] End2end demo of KIE @innerlee

[amitbcp]

Can we include support for Transfer Learning for KIE ? Like we load the pretrained KIE model or different UNET & Bi-LSTM and train the GLCN or just linear layer for node classification ?

[innerlee]

@amitbcp Will a flag that can freeze unet parameters be good enough?

[amitbcp]

@innerlee , there are actually two use cases, we can prioritize accordingly :

1. Fine Tune only the last linear layer with custom classes. This can be the last FC or last two FC. Checkpoint can be from KIE
2. Load different checkpoints for UNET, BI-LSTM , GLN and continue training on custom classes

I guess a Flag for each major component and each model block might be good. But just having a flag for each block would be great

[amitbcp]

Another request would be to integrate the model graphs with TensorBoard Logging in the config so that it becomes more intuitive to understand the model architecture. Currently the TensorBoardHook only supports Logging scalars

[gaotongxiao]

@amitbcp It's a cool idea but it might be better to request such a hook in MMCV's repo. My colleague @zhouzaida said they have this in their future plan.

[amitbcp]

ohh that's great @gaotongxiao ! S should I create a ticket there or is it fine as @zhouzaida already has it in the plan ?

[gaotongxiao]

@amitbcp It would be great to create a ticket there

[amitbcp]

Hi @gaotongxiao , can we include this in the upcoming release : https://github.com/open-mmlab/mmocr/issues/248

It will usher more research in the Document AI domain if users can explore the framework in better details

[gaotongxiao]

@amitbcp Sure, we have been working on it and will release it soon.

[anuj-rathore]

@innerlee @yuexy How can we convert KIE PyTorch model to ONNX?