CRNN ValueError: Tried to convert 'x' to a tensor and failed. Error: Cannot create a tensor proto whose content is larger than 2GB.

Traceback (most recent call last): File "train.py", line 95, in training_model.fit_generator(train_generator, steps_per_epoch=int(train_generator.nb_samples / train_generator.batch_size), epochs=cfg.nb_epochs, verbose=1, workers=cfg.nb_workers, use_multiprocessing=True, callbacks=callbacks) File "/home/sherry/.local/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/home/sherry/.local/lib/python3.6/site-packages/keras/engine/training.py", line 1418, in fit_generator initial_epoch=initial_epoch) File "/home/sherry/.local/lib/python3.6/site-packages/keras/engine/training_generator.py", line 223, in fit_generator callbacks.on_batch_end(batch_index, batch_logs) File "/home/sherry/.local/lib/python3.6/site-packages/keras/callbacks.py", line 115, in on_batch_end callback.on_batch_end(batch, logs) File "/home/sherry/CRNN-master/utils.py", line 130, in on_batch_end accuracy, correct_char_predictions = self.evaluate() File "/home/sherry/CRNN-master/utils.py", line 162, in evaluate ctc_decode = K.ctc_decode(y_pred[:, 2:, :], input_length=np.ones(shape[0])*shape[1])[0][0] File "/home/sherry/.local/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 4289, in ctc_decode y_pred = tf.log(tf.transpose(y_pred, perm=[1, 0, 2]) + epsilon()) File "/usr/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1420, in transpose ret = transpose_fn(a, perm, name=name) File "/usr/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 8927, in transpose "Transpose", x=x, perm=perm, name=name) File "/usr/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 528, in _apply_op_helper (input_name, err))

how to solve the problem?

Aug 22 '19 07:08 lxyzler

What are the specs of the machine you use? Particularly the GPU. And tell me your TF and Keras version too, please.

Aug 29 '19 05:08 janzd

Linux xiaoi2018-754 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

tf.version '1.12.0'

print(keras.version) 2.2.4

by the way ，I also want to ask what your annotation_train.txt file looks like.Can you give me a example,I use my own dataset,

22240671_1034114836.jpg 35 50 27 45 637 1543 210 10 431 426

46465187_719612387.jpg 207 20 206 554 58 314 36 99 335 255

24991671_3672543637.jpg 114 949 50 14 527 415 330 2 981 833

32864234_3953211135.jpg 2 1495 607 3216 736 931 1495 493 435 28

69056359_2721077874.jpg 114 556 8 82 728 28 41 109 31 162

62780546_3631858873.jpg 12 295 25 227 165 13 207 20 368 1556

21057171_2803761122.jpg 22 19 60 253 380 1039 30 174 249 198

28059718_3439131428.jpg 455 18 12 195 80 1421 13 1 119 20

36108140_2080626420.jpg 6 76 94 675 76 19 6 6 19 46

this is my txt,each number corresponds to a Chinese character.Is there any problem with this format?

look forward to your kind reply/advice

------------------ 原始邮件 ------------------ 发件人: "Jan Zdenek"[email protected]; 发送时间: 2019年8月29日(星期四) 中午1:07 收件人: "kurapan/CRNN"[email protected]; 抄送: "kk"[email protected];"Author"[email protected]; 主题: Re: [kurapan/CRNN] ValueError: Tried to convert 'x' to a tensor andfailed. Error: Cannot create a tensor proto whose content is larger than 2GB.(#4)

What are the specs of the machine you use? Particularly the GPU. And tell me your TF and Keras version too, please.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

Aug 29 '19 05:08 lxyzler

I haven't tested it with TF 1.12 and Keras version higher than 2.2, but I think it should work with those versions too. How large is your input? This error indicates that it tried to initialize a variable larger than 2 GB, which doesn't work by default. https://stackoverflow.com/questions/51470991/create-a-tensor-proto-whose-content-is-larger-than-2gb

It uses Synth90k annotation. There is a file, let's call it annotation.txt, that contains paths to images and the IDs of the words in those images. Example:

img/wither.jpg 4
img/illager.jpg 1
img/ENDER.jpg 2
img/vindicator.jpg 5
img/PILLAGER.jpg 0
img/evoker.jpg 3

Then, you have a file lexicon.txt where all words (in your case sequences of Chinese characters) are stored. The above-mentioned IDs correspond to line numbers in this file (the first line is 0).

PILLAGER
illager
ENDER
evoker
wither
vindicator

If you'd rather use your format, which I like better than the Synth90k format actually, you'd have rewrite the data loading part of the code.

Aug 30 '19 10:08 janzd

I have rewrite the data_generator File "/home/sherry/CRNN-master/utils.py", line 162, in evaluate ctc_decode = K.ctc_decode(y_pred[:, 2:, :], input_length=np.ones(shape[0])*shape[1])[0][0]

I try the Synth90k,in this case,y_pred[:, 2:, :].shape is (5000, 48, 37) if l use my dataset,the shape is (5000, 48, 5990),because l have 5990 Chinese characters Does this cause problems?y_pred[:, 2:, :] can not be too large?

Aug 31 '19 09:08 lxyzler

CRNN CRNN copied to clipboard

ValueError: Tried to convert 'x' to a tensor and failed. Error: Cannot create a tensor proto whose content is larger than 2GB.

CRNN
CRNN copied to clipboard