Size mismatch error Fine Tuning latin_g2

[emigomez]

Hello,

I'm trying to fine-tune the latin_g2 model (based on your script on the /trainer folder), but I'm having errors in the execution. My first step was to finetune the english_g2 model, with that I hadn't any problems in the training but also none during the inference.

This is the error at the beginning of the fine tuning with the latin_g2 model (I also tried with latin model):

`........................
dataset_root:    all_data/dni_dataset/val        dataset: /
all_data/dni_dataset/val/
sub-directory:  /.       num samples: 408
No Transformation module specified
model input parameters 64 600 20 1 256 256 407 34 None VGG BiLSTM CTC
loading pretrained model from saved_models/latin_g2.pth
Traceback (most recent call last):
  File "trainer_esp.py", line 32, in <module>
    train(opt, amp=False)
  File "C:\Users\MSI\Desktop\INETUM\INETUM_Projects\OCR\easyocr\finetuning_easyocr\train.py", line 77, in train
    model.load_state_dict(pretrained_dict, strict=False)
  File "C:\Users\MSI\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py", line 1482, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DataParallel:
        size mismatch for module.Prediction.weight: copying a param with shape torch.Size([352, 256]) from checkpoint, the shape in current model is torch.Size([407, 256]).
        size mismatch for module.Prediction.bias: copying a param with shape torch.Size([352]) from checkpoint, the shape in current model is torch.Size([407]).`

I think that the error is because I have set badly the symbol and lang_char parameter of the yml file for this latin_g2 model. I tried with several options like the original one use for english, antoher lists write by my and the list specified for this model in https://github.com/JaidedAI/EasyOCR/blob/master/easyocr/config.py#L149. But the result for all of this test is the same, the above error.

This is the .yml file that I have used for the fine tuning:

`number: '0123456789'
symbol: " !\"#$%&'()*+,-./0123456789:;<=>?@[\\]^_`{|}~ €" 
lang_char: " !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~ªÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿĀāĂ㥹ĆćČčĎďĐđĒēĖėĘęĚěĞğĨĩĪīĮįİıĶķĹĺĻļĽľŁłŃńŅņŇňŒœŔŕŘřŚśŞşŠšŤťŨũŪūŮůŲųŸŹźŻżŽžƏƠơƯưȘșȚțə̇ḌḍḶḷṀṁṂṃṄṅṆṇṬṭẠạẢảẤấẦầẨẩẪẫẬậẮắẰằẲẳẴẵẶặẸẹẺẻẼẽẾếỀềỂểỄễỆệỈỉỊịỌọỎỏỐốỒồỔổỖỗỘộỚớỜờỞởỠỡỢợỤụỦủỨứỪừỬửỮữỰựỲỳỴỵỶỷỸỹ€" 
# lang_char: "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzªÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿĀāĂ㥹ĆćČčĎďĐđĒēĖėĘęĚěĞğĨĩĪīĮįİıĶķĹĺĻļĽľŁłŃńŅņŇňŒœŔŕŘřŚśŞşŠšŤťŨũŪūŮůŲųŸŹźŻżŽžƏƠơƯưȘșȚțə̇ḌḍḶḷṀṁṂṃṄṅṆṇṬṭẠạẢảẤấẦầẨẩẪẫẬậẮắẰằẲẳẴẵẶặẸẹẺẻẼẽẾếỀềỂểỄễỆệỈỉỊịỌọỎỏỐốỒồỔổỖỗỘộỚớỜờỞởỠỡỢợỤụỦủỨứỪừỬửỮữỰựỲỳỴỵỶỷỸỹ"
# lang_char:  'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'
experiment_name: 'esp_dni'
train_data: 'all_data/dni_dataset
valid_data: 'all_data/dni_dataset/val'
manualSeed: 1111
workers: 1 
batch_size: 8 
num_iter: 25000 
valInterval: 20000
saved_model: 'saved_models/latin_g2.pth' 
FT: True 
optim: False # default is Adadelta
lr: 1.
beta1: 0.9
rho: 0.95
eps: 0.00000001
grad_clip: 5
#Data processing
select_data: 'all_data/dni_dataset' 
batch_ratio: '1' 
total_data_usage_ratio: 1.0
batch_max_length: 34 
imgH: 64
imgW: 600
rgb: False
contrast_adjust: False
sensitive: True
PAD: True
contrast_adjust: 0.0
data_filtering_off: False
#Model Architecture
Transformation: 'None'
FeatureExtraction: 'VGG'
SequenceModeling: 'BiLSTM'
Prediction: 'CTC'
num_fiducial: 20
input_channel: 1
output_channel: 256
hidden_size: 256
decode: 'greedy'
new_prediction: False
freeze_FeatureFxtraction: False
freeze_SequenceModeling: False`

In that sense I have some questions:

• Is the error related with these parameters? symbol and lang_char
• Is it possible to decide on my own char list?
• Is it mandatory to used the char list of https://github.com/JaidedAI/EasyOCR/blob/master/easyocr/config.py#L149. Because lots of characters from the latin are not useful for me like ẻẼẽẾếỀềỂểỄễỆệỈỉỊịỌọỎỏỐốỒồỔổỖỗỘộỚớỜờỞởỠỡỢợỤụỦủỨứỪừỬửỮữỰựỲỳỴỵỶỷỸỹ€

Thank you for your help

[emigomez]

I have install easyocr with pip, I don't know if maybe this is the problem and i have to installed with this repo from source

[rkcosmos]

To train new set of characters, you can set new_prediction to True.

[SkygirlLuna]

What's the effect of set new_prediction to True? Can I still do a finetuning of the latin model or does it means that a do a complete retraining?

[santiac89]

@emigomez I'm pretty sure this is due to the shape of the network, I used:

input_channel = 1 output_channel = 256 hidden_size = 256

And it gets past the error, but now I'm having trouble with the character set, I don't know what to use for lating_g2 but I get this error

UnicodeEncodeError: 'charmap' codec can't encode characters in position 654-701: character maps to <undefined>

[khawar-islam]

@emigomez Are you able to solve the mismatch error? I am getting same error

[mohamedsouguir]

Is there a solution for the mismatch error please?

[RishabhSheoran]

Hello! Did anyone find a solution for the mismatch error? I'm facing the same for english_g2.pth model and I'm following this article to finetune: https://pub.towardsai.net/how-to-fine-tune-easyocr-to-achieve-better-ocr-performance-1540f5076428