PICK-pytorch
PICK-pytorch copied to clipboard
wenwenyu
About SROIE Dataset Preparation
Hi, that's a good work on IE, thanks.
Currently, I've tested your code on SROIE with the "document_level" setting. I used the OCR results to train and test the model, which downloaded from SROIE official website and extracted from the folder "task1&2_train(626p)" and "taks1&2_text_test(361p)". The performance looks not good as yours. May I ask how you prepared your dataset. Did you use other OCR tools to preprocess the dataset.
Thank you very much.
I've did some changes on original code and added some post-processing of output results. The set used 'box_and_within_box_level'. Now the performance only reached to around 88%.
Hope someone who successfully reimplemented the result in paper can give me some help, thank you.
@ning-mz : If you don't mind can you give some hint about post processing?
Hi, basically it was depends on the output result. The code provided now is not exactly suit for the official SROIE mission.
For example, sometimes it has multiple outputs on the entity "Total", I chose the one with highest confidence mark. Meanwhile, for the output of "Address", you need to add spaces into result, cause the output of PICK is a sequence, it ignored the \n in receipts. For example:
Department of Computer Science, University of Liverpool, Liverpool
The output of PICK now will be: Department of Computer Science,University of Liverpool,Liverpool The correct result should be: Department of Computer Science, University of Liverpool, Liverpool Note the space after ,
Meanwhile, due to the original data preprocess in code, it will drop the full stop . at the end of sentence especially in "Company", like: XXX Co. Ltd. it will drop the last full stop and process it like: XXX Co. Ltd
And the official SROIE also has OCR mismatching, it will also influence the performance.
It is simple to add some rules with evaluate the results. Good luck!
@ning-mz
For example, sometimes it has multiple outputs on the entity "Total", I chose the one with highest confidence mark.
Please can you share how you did it? I have not managed to obtain a confidence value for each output.
Thanks
@ning-mz @ninjakx Did you get any good way to deal this problem? (I wondered how to extract multiple data in one entity(like product price in SROIE dataset.) Could you please share some tips? @wenwenyu
