post_ocr_correction icon indicating copy to clipboard operation
post_ocr_correction copied to clipboard

How can I train the model with the dataset in the paper?

Open Alexsong111 opened this issue 2 years ago • 4 comments

If I run the script “Train a model from scratch”, it does not use the dataset in the paper.

I hope to use your dataset to train your model again to get a same trained model with yours. Thanks!

Alexsong111 avatar Jul 31 '23 21:07 Alexsong111

The code in Train a new model from scratch is to create new models, the first two lines are some dummy data to train the model on. You can load the models developed in the paper by looking at the code in Load one of the pre-trained models:

image

If you want to do all the training all over again, you need to look into the notebooks folder, which contains the step-by-step process of how the models were trained. Unfortunately, it's a bit outdated, so you will need to do some minor modifications.

jarobyte91 avatar Aug 02 '23 18:08 jarobyte91

You can also look at previous issues to see if that's what your problem is:

https://github.com/jarobyte91/post_ocr_correction/issues?q=is%3Aissue+is%3Aclosed

jarobyte91 avatar Aug 02 '23 18:08 jarobyte91

The code in Train a new model from scratch is to create new models, the first two lines are some dummy data to train the model on. You can load the models developed in the paper by looking at the code in Load one of the pre-trained models:

image

If you want to do all the training all over again, you need to look into the notebooks folder, which contains the step-by-step process of how the models were trained. Unfortunately, it's a bit outdated, so you will need to do some minor modifications.

Hi, I tried to update the code in notebook before, but encounter some difficulties. Could you please update the code in notebook?

Alexsong111 avatar Aug 02 '23 20:08 Alexsong111

What difficulties specifically? You need to replace the following:

pytorch_decoding --> pytorch_beam_search ocr_correction --> post_ocr_correction

These are the old names of the packages

jarobyte91 avatar Aug 07 '23 19:08 jarobyte91