CRNN
CRNN copied to clipboard
Belval
The loss always be inf
I trained with the data generated by your tool TextRecognitionDataGenerator. And trained with 100 iterations, the loss is always be inf, I'm wondering the reason, thx a lot.
Hi, inf loss does happen from time to time but usually, it's either a random occurrence (so restarting the training will fix it) or a problem with the dataset.
How big is your dataset? Are you training for Chinese? If so, what is the length of your CHAR_VECTOR?
Thank you
Thanks for your immediate reply. Tried to restart the training many times, without any improvement. BTW, the size of my training dataset is 8000 in english. I just added some punctuations like ,.;'" to the CHAR_VECTOR.
Interesting, I'll do some testing from my side see if I can get a test case.
If you could compress your 8000 samples dataset and send it to me to ensure reproducibility I would be grateful.
You can drop the file here: https://nextcloud.belval.org/index.php/s/MQsQrK8egatQtLo
Sure, I'm glad to do that, the size is 10001 actually. And my CHAR_VECTOR is
CHAR_VECTOR = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ-'.!?,\"" just reuse yours.
Just a heads up to say that I was able to reproduce the issue. I'm quite busy at the moment but I'll try to troubleshoot it ASAP.
Thank you for bringing this to my attention.
My dataset is Chinese and the length of CHAR_VECTOR is 70000+, the long lengths can cause problems?
My dataset is Chinese and the length of CHAR_VECTOR is 70000+, the long lengths can cause problems?
sorry, my CHAR_VECTOR is 7000+.
The training process is too slow. How can I solve it? It trains five steps in four days.
I face the same problem. Have you solved it?
Not yet. I used this project without this problem https://github.com/wushilian/STN_CNN_LSTM_CTC_TensorFlow/blob/master/utils.py
I face the same problem. Have you solved it?
Not yet. I used this project without this problem https://github.com/wushilian/STN_CNN_LSTM_CTC_TensorFlow/blob/master/utils.py
Thank you. I'll try.