mmocr icon indicating copy to clipboard operation
mmocr copied to clipboard

[Refactor] Refactor data converter and gather

Open Harold-lkk opened this issue 2 years ago • 1 comments

主要有以下更新:

  1. 将 Gatherer 方法抽象成单独模块
  2. 将 DataConverter 移除,其中的 pack_instance 抽象成 Packer
  3. 将流程更改为
for split in splits
    self.obtainer()
    self.gatherer()
    self.parser()
    self.packer()
    self.dumper()

细微改动:

  1. 规范 parser 的输入接口和输出接口:
  • parse_files 输入统一为 img_paths, ann_paths:
    • 对于一个json 文件包含全部标注: img_paths 图像文件夹路径,ann_path 对应标注文件路径
    • 对于多个图像对应多个文件:img_paths 为图像路径列表,ann_paths 为对应图像列表
  • 输出统一为list[tuple(图像路径,解析好的标注信息)]
  1. Obtainer 增加了是否解压的判定,解压完成后增加一个 finsh.mmocr 文件

Harold-lkk avatar Feb 06 '23 14:02 Harold-lkk

Invite @KevinNuNu to take a look

gaotongxiao avatar Feb 21 '23 11:02 gaotongxiao

Codecov Report

Patch coverage: 75.47% and project coverage change: +2.04 :tada:

Comparison is base (62d440f) 88.08% compared to head (1c1cb07) 90.13%.

:exclamation: Current head 1c1cb07 differs from pull request most recent head 6bee8c0. Consider uploading reports for the commit 6bee8c0 to get more accurate results

Additional details and impacted files
@@             Coverage Diff             @@
##           dev-1.x    #1707      +/-   ##
===========================================
+ Coverage    88.08%   90.13%   +2.04%     
===========================================
  Files          176      190      +14     
  Lines        11022    11128     +106     
  Branches      1558     1567       +9     
===========================================
+ Hits          9709    10030     +321     
+ Misses        1022      790     -232     
- Partials       291      308      +17     
Flag Coverage Δ
unittests 90.13% <75.47%> (+2.04%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...atasets/preparers/obtainers/naive_data_obtainer.py 16.82% <5.88%> (ø)
...r/datasets/preparers/packers/wildreceipt_packer.py 16.36% <16.36%> (ø)
mmocr/datasets/preparers/gatherers/naf_gatherer.py 31.42% <31.42%> (ø)
mmocr/datasets/preparers/parsers/coco_parser.py 18.00% <33.33%> (+0.69%) :arrow_up:
mmocr/datasets/preparers/parsers/base.py 80.00% <77.77%> (+2.22%) :arrow_up:
mmocr/utils/processing.py 79.31% <79.31%> (ø)
mmocr/datasets/preparers/dumpers/base.py 80.00% <80.00%> (ø)
mmocr/datasets/preparers/packers/base.py 81.25% <81.25%> (ø)
...ocr/datasets/preparers/packers/textrecog_packer.py 81.25% <81.25%> (ø)
mmocr/datasets/preparers/data_preparer.py 83.67% <81.81%> (+15.49%) :arrow_up:
... and 25 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov[bot] avatar Feb 28 '23 16:02 codecov[bot]