EAST icon indicating copy to clipboard operation
EAST copied to clipboard

Training issue

Open hnn123 opened this issue 6 years ago • 5 comments

When I run the multigpu_train.py in win10, I have encounter a problem as the following

_### Generator use 10 batches for buffering, this may take a while, you can tune this yourself. Exception in thread Thread-3: Traceback (most recent call last): File "C:\Users\huang\AppData\Local\Programs\Python\Python35\lib\threading.py", line 914, in _bootstrap_inner self.run() File "C:\Users\huang\AppData\Local\Programs\Python\Python35\lib\threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "D:\tensorflow\ocr\EAST-master\EAST-master\data_util.py", line 53, in data_generator_task generator_output = next(self._generator) ValueError: generator already executing

1000 training images in data/tianchi/ Traceback (most recent call last): File "multigpu_train.py", line 180, in tf.app.run() File "C:\Users\huang\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\platform\app.py", line 126, in run sys.exit(main(argv)) File "multigpu_train.py", line 154, in main ml, tl, _ = sess.run([model_loss, total_loss, train_op], feed_dict={input_images: data[0], TypeError: 'NoneType' object is not subscriptable poly in wrong direction -1.1246908 [187.2 104.8] [177.43877 84.39016] poly in wrong direction poly in wrong direction poly in wrong direction

Did anyone encounter the same issue?

hnn123 avatar May 19 '18 04:05 hnn123

Check the order of coordinates, it should be like ['x', 'y', 'x+width', 'y', 'x+width', 'y+height', 'x', 'y+height']

vsd-dev avatar Jun 04 '18 06:06 vsd-dev

i have a same error,and I found the reason.

python multigpu_train.py --gpu_list=0 --input_size=512 --batch_size_per_gpu=8 --checkpoint_path=./models/east_icpr2018_resnet_v1_50_rbox_1035k/ --text_scale=512 --training_data_path=./data/ICPR2018/ --geometry=RBOX --learning_rate=0.0001 --num_readers=10 --max_steps=50000

================= --num_readers=10 ,is too large to cause thread run failure, i set like --num_readers=1,the error is solved

mm1860 avatar Aug 14 '18 17:08 mm1860

Hello, I met the same problem, have you solved the problem, please help me, thank you very much.

Shiyi19971017 avatar Dec 23 '19 06:12 Shiyi19971017

hello @hnn123 I used the icpr mtwi 2018 dataset and encountered the same problem. Have you solved problem? How should I solve it?please help me,thank you!

qingqing625 avatar Dec 26 '19 09:12 qingqing625

Hello, I have only temporarily solved this error by reducing the number of training sessions.However, this problem will still occur if I set more training times. If I am not in a hurry, I can first reduce the training times and try the training effect. I hope it can help you.By the way, I am also studying and can communicate with each other if I have any questions.

Shiyi19971017 avatar Dec 26 '19 10:12 Shiyi19971017