EAST icon indicating copy to clipboard operation
EAST copied to clipboard

EOFError when training

Open jhaggle opened this issue 4 years ago • 0 comments

Starting the training I get the EOFError shown below. What am I doing wrong?

(keras2) johan@johan-VirtualBox:~/Documents/school_projects/temp/EAST-master$ python3 train.py --gpu_list=1 --input_size=512 --batch_size=12 --nb_workers=6 --training_data_path=/home/johan/Documents/school_projects/temp/train_data40 --validation_data_path=/home/johan/Documents/school_projects/temp/train_data40 --checkpoint_path=/home/johan/Documents/school_projects/temp/EAST-master/temp --max_epochs=3 /home/johan/virtualenvs/keras2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:471: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /home/johan/virtualenvs/keras2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:472: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /home/johan/virtualenvs/keras2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:473: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /home/johan/virtualenvs/keras2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:474: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /home/johan/virtualenvs/keras2/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:475: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) Using TensorFlow backend. Number of validation images : 40 Training with 1 GPU W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations. W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations. W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.


Layer (type) Output Shape Param # Connected to

input_image (InputLayer) (None, None, None, 3 0


conv1_pad (ZeroPadding2D) (None, None, None, 3 0 input_image[0][0]


conv1 (Conv2D) (None, None, None, 6 9472 conv1_pad[0][0]


bn_conv1 (BatchNormalization) (None, None, None, 6 256 conv1[0][0]


activation_1 (Activation) (None, None, None, 6 0 bn_conv1[0][0]


max_pooling2d_1 (MaxPooling2D) (None, None, None, 6 0 activation_1[0][0]


res2a_branch2a (Conv2D) (None, None, None, 6 4160 max_pooling2d_1[0][0]


bn2a_branch2a (BatchNormalizati (None, None, None, 6 256 res2a_branch2a[0][0]


activation_2 (Activation) (None, None, None, 6 0 bn2a_branch2a[0][0]


res2a_branch2b (Conv2D) (None, None, None, 6 36928 activation_2[0][0]


bn2a_branch2b (BatchNormalizati (None, None, None, 6 256 res2a_branch2b[0][0]


activation_3 (Activation) (None, None, None, 6 0 bn2a_branch2b[0][0]


res2a_branch2c (Conv2D) (None, None, None, 2 16640 activation_3[0][0]


res2a_branch1 (Conv2D) (None, None, None, 2 16640 max_pooling2d_1[0][0]


bn2a_branch2c (BatchNormalizati (None, None, None, 2 1024 res2a_branch2c[0][0]


bn2a_branch1 (BatchNormalizatio (None, None, None, 2 1024 res2a_branch1[0][0]


add_1 (Add) (None, None, None, 2 0 bn2a_branch2c[0][0]
bn2a_branch1[0][0]


activation_4 (Activation) (None, None, None, 2 0 add_1[0][0]


res2b_branch2a (Conv2D) (None, None, None, 6 16448 activation_4[0][0]


bn2b_branch2a (BatchNormalizati (None, None, None, 6 256 res2b_branch2a[0][0]


activation_5 (Activation) (None, None, None, 6 0 bn2b_branch2a[0][0]


res2b_branch2b (Conv2D) (None, None, None, 6 36928 activation_5[0][0]


bn2b_branch2b (BatchNormalizati (None, None, None, 6 256 res2b_branch2b[0][0]


activation_6 (Activation) (None, None, None, 6 0 bn2b_branch2b[0][0]


res2b_branch2c (Conv2D) (None, None, None, 2 16640 activation_6[0][0]


bn2b_branch2c (BatchNormalizati (None, None, None, 2 1024 res2b_branch2c[0][0]


add_2 (Add) (None, None, None, 2 0 bn2b_branch2c[0][0]
activation_4[0][0]


activation_7 (Activation) (None, None, None, 2 0 add_2[0][0]


res2c_branch2a (Conv2D) (None, None, None, 6 16448 activation_7[0][0]


bn2c_branch2a (BatchNormalizati (None, None, None, 6 256 res2c_branch2a[0][0]


activation_8 (Activation) (None, None, None, 6 0 bn2c_branch2a[0][0]


res2c_branch2b (Conv2D) (None, None, None, 6 36928 activation_8[0][0]


bn2c_branch2b (BatchNormalizati (None, None, None, 6 256 res2c_branch2b[0][0]


activation_9 (Activation) (None, None, None, 6 0 bn2c_branch2b[0][0]


res2c_branch2c (Conv2D) (None, None, None, 2 16640 activation_9[0][0]


bn2c_branch2c (BatchNormalizati (None, None, None, 2 1024 res2c_branch2c[0][0]


add_3 (Add) (None, None, None, 2 0 bn2c_branch2c[0][0]
activation_7[0][0]


activation_10 (Activation) (None, None, None, 2 0 add_3[0][0]


res3a_branch2a (Conv2D) (None, None, None, 1 32896 activation_10[0][0]


bn3a_branch2a (BatchNormalizati (None, None, None, 1 512 res3a_branch2a[0][0]


activation_11 (Activation) (None, None, None, 1 0 bn3a_branch2a[0][0]


res3a_branch2b (Conv2D) (None, None, None, 1 147584 activation_11[0][0]


bn3a_branch2b (BatchNormalizati (None, None, None, 1 512 res3a_branch2b[0][0]


activation_12 (Activation) (None, None, None, 1 0 bn3a_branch2b[0][0]


res3a_branch2c (Conv2D) (None, None, None, 5 66048 activation_12[0][0]


res3a_branch1 (Conv2D) (None, None, None, 5 131584 activation_10[0][0]


bn3a_branch2c (BatchNormalizati (None, None, None, 5 2048 res3a_branch2c[0][0]


bn3a_branch1 (BatchNormalizatio (None, None, None, 5 2048 res3a_branch1[0][0]


add_4 (Add) (None, None, None, 5 0 bn3a_branch2c[0][0]
bn3a_branch1[0][0]


activation_13 (Activation) (None, None, None, 5 0 add_4[0][0]


res3b_branch2a (Conv2D) (None, None, None, 1 65664 activation_13[0][0]


bn3b_branch2a (BatchNormalizati (None, None, None, 1 512 res3b_branch2a[0][0]


activation_14 (Activation) (None, None, None, 1 0 bn3b_branch2a[0][0]


res3b_branch2b (Conv2D) (None, None, None, 1 147584 activation_14[0][0]


bn3b_branch2b (BatchNormalizati (None, None, None, 1 512 res3b_branch2b[0][0]


activation_15 (Activation) (None, None, None, 1 0 bn3b_branch2b[0][0]


res3b_branch2c (Conv2D) (None, None, None, 5 66048 activation_15[0][0]


bn3b_branch2c (BatchNormalizati (None, None, None, 5 2048 res3b_branch2c[0][0]


add_5 (Add) (None, None, None, 5 0 bn3b_branch2c[0][0]
activation_13[0][0]


activation_16 (Activation) (None, None, None, 5 0 add_5[0][0]


res3c_branch2a (Conv2D) (None, None, None, 1 65664 activation_16[0][0]


bn3c_branch2a (BatchNormalizati (None, None, None, 1 512 res3c_branch2a[0][0]


activation_17 (Activation) (None, None, None, 1 0 bn3c_branch2a[0][0]


res3c_branch2b (Conv2D) (None, None, None, 1 147584 activation_17[0][0]


bn3c_branch2b (BatchNormalizati (None, None, None, 1 512 res3c_branch2b[0][0]


activation_18 (Activation) (None, None, None, 1 0 bn3c_branch2b[0][0]


res3c_branch2c (Conv2D) (None, None, None, 5 66048 activation_18[0][0]


bn3c_branch2c (BatchNormalizati (None, None, None, 5 2048 res3c_branch2c[0][0]


add_6 (Add) (None, None, None, 5 0 bn3c_branch2c[0][0]
activation_16[0][0]


activation_19 (Activation) (None, None, None, 5 0 add_6[0][0]


res3d_branch2a (Conv2D) (None, None, None, 1 65664 activation_19[0][0]


bn3d_branch2a (BatchNormalizati (None, None, None, 1 512 res3d_branch2a[0][0]


activation_20 (Activation) (None, None, None, 1 0 bn3d_branch2a[0][0]


res3d_branch2b (Conv2D) (None, None, None, 1 147584 activation_20[0][0]


bn3d_branch2b (BatchNormalizati (None, None, None, 1 512 res3d_branch2b[0][0]


activation_21 (Activation) (None, None, None, 1 0 bn3d_branch2b[0][0]


res3d_branch2c (Conv2D) (None, None, None, 5 66048 activation_21[0][0]


bn3d_branch2c (BatchNormalizati (None, None, None, 5 2048 res3d_branch2c[0][0]


add_7 (Add) (None, None, None, 5 0 bn3d_branch2c[0][0]
activation_19[0][0]


activation_22 (Activation) (None, None, None, 5 0 add_7[0][0]


res4a_branch2a (Conv2D) (None, None, None, 2 131328 activation_22[0][0]


bn4a_branch2a (BatchNormalizati (None, None, None, 2 1024 res4a_branch2a[0][0]


activation_23 (Activation) (None, None, None, 2 0 bn4a_branch2a[0][0]


res4a_branch2b (Conv2D) (None, None, None, 2 590080 activation_23[0][0]


bn4a_branch2b (BatchNormalizati (None, None, None, 2 1024 res4a_branch2b[0][0]


activation_24 (Activation) (None, None, None, 2 0 bn4a_branch2b[0][0]


res4a_branch2c (Conv2D) (None, None, None, 1 263168 activation_24[0][0]


res4a_branch1 (Conv2D) (None, None, None, 1 525312 activation_22[0][0]


bn4a_branch2c (BatchNormalizati (None, None, None, 1 4096 res4a_branch2c[0][0]


bn4a_branch1 (BatchNormalizatio (None, None, None, 1 4096 res4a_branch1[0][0]


add_8 (Add) (None, None, None, 1 0 bn4a_branch2c[0][0]
bn4a_branch1[0][0]


activation_25 (Activation) (None, None, None, 1 0 add_8[0][0]


res4b_branch2a (Conv2D) (None, None, None, 2 262400 activation_25[0][0]


bn4b_branch2a (BatchNormalizati (None, None, None, 2 1024 res4b_branch2a[0][0]


activation_26 (Activation) (None, None, None, 2 0 bn4b_branch2a[0][0]


res4b_branch2b (Conv2D) (None, None, None, 2 590080 activation_26[0][0]


bn4b_branch2b (BatchNormalizati (None, None, None, 2 1024 res4b_branch2b[0][0]


activation_27 (Activation) (None, None, None, 2 0 bn4b_branch2b[0][0]


res4b_branch2c (Conv2D) (None, None, None, 1 263168 activation_27[0][0]


bn4b_branch2c (BatchNormalizati (None, None, None, 1 4096 res4b_branch2c[0][0]


add_9 (Add) (None, None, None, 1 0 bn4b_branch2c[0][0]
activation_25[0][0]


activation_28 (Activation) (None, None, None, 1 0 add_9[0][0]


res4c_branch2a (Conv2D) (None, None, None, 2 262400 activation_28[0][0]


bn4c_branch2a (BatchNormalizati (None, None, None, 2 1024 res4c_branch2a[0][0]


activation_29 (Activation) (None, None, None, 2 0 bn4c_branch2a[0][0]


res4c_branch2b (Conv2D) (None, None, None, 2 590080 activation_29[0][0]


bn4c_branch2b (BatchNormalizati (None, None, None, 2 1024 res4c_branch2b[0][0]


activation_30 (Activation) (None, None, None, 2 0 bn4c_branch2b[0][0]


res4c_branch2c (Conv2D) (None, None, None, 1 263168 activation_30[0][0]


bn4c_branch2c (BatchNormalizati (None, None, None, 1 4096 res4c_branch2c[0][0]


add_10 (Add) (None, None, None, 1 0 bn4c_branch2c[0][0]
activation_28[0][0]


activation_31 (Activation) (None, None, None, 1 0 add_10[0][0]


res4d_branch2a (Conv2D) (None, None, None, 2 262400 activation_31[0][0]


bn4d_branch2a (BatchNormalizati (None, None, None, 2 1024 res4d_branch2a[0][0]


activation_32 (Activation) (None, None, None, 2 0 bn4d_branch2a[0][0]


res4d_branch2b (Conv2D) (None, None, None, 2 590080 activation_32[0][0]


bn4d_branch2b (BatchNormalizati (None, None, None, 2 1024 res4d_branch2b[0][0]


activation_33 (Activation) (None, None, None, 2 0 bn4d_branch2b[0][0]


res4d_branch2c (Conv2D) (None, None, None, 1 263168 activation_33[0][0]


bn4d_branch2c (BatchNormalizati (None, None, None, 1 4096 res4d_branch2c[0][0]


add_11 (Add) (None, None, None, 1 0 bn4d_branch2c[0][0]
activation_31[0][0]


activation_34 (Activation) (None, None, None, 1 0 add_11[0][0]


res4e_branch2a (Conv2D) (None, None, None, 2 262400 activation_34[0][0]


bn4e_branch2a (BatchNormalizati (None, None, None, 2 1024 res4e_branch2a[0][0]


activation_35 (Activation) (None, None, None, 2 0 bn4e_branch2a[0][0]


res4e_branch2b (Conv2D) (None, None, None, 2 590080 activation_35[0][0]


bn4e_branch2b (BatchNormalizati (None, None, None, 2 1024 res4e_branch2b[0][0]


activation_36 (Activation) (None, None, None, 2 0 bn4e_branch2b[0][0]


res4e_branch2c (Conv2D) (None, None, None, 1 263168 activation_36[0][0]


bn4e_branch2c (BatchNormalizati (None, None, None, 1 4096 res4e_branch2c[0][0]


add_12 (Add) (None, None, None, 1 0 bn4e_branch2c[0][0]
activation_34[0][0]


activation_37 (Activation) (None, None, None, 1 0 add_12[0][0]


res4f_branch2a (Conv2D) (None, None, None, 2 262400 activation_37[0][0]


bn4f_branch2a (BatchNormalizati (None, None, None, 2 1024 res4f_branch2a[0][0]


activation_38 (Activation) (None, None, None, 2 0 bn4f_branch2a[0][0]


res4f_branch2b (Conv2D) (None, None, None, 2 590080 activation_38[0][0]


bn4f_branch2b (BatchNormalizati (None, None, None, 2 1024 res4f_branch2b[0][0]


activation_39 (Activation) (None, None, None, 2 0 bn4f_branch2b[0][0]


res4f_branch2c (Conv2D) (None, None, None, 1 263168 activation_39[0][0]


bn4f_branch2c (BatchNormalizati (None, None, None, 1 4096 res4f_branch2c[0][0]


add_13 (Add) (None, None, None, 1 0 bn4f_branch2c[0][0]
activation_37[0][0]


activation_40 (Activation) (None, None, None, 1 0 add_13[0][0]


res5a_branch2a (Conv2D) (None, None, None, 5 524800 activation_40[0][0]


bn5a_branch2a (BatchNormalizati (None, None, None, 5 2048 res5a_branch2a[0][0]


activation_41 (Activation) (None, None, None, 5 0 bn5a_branch2a[0][0]


res5a_branch2b (Conv2D) (None, None, None, 5 2359808 activation_41[0][0]


bn5a_branch2b (BatchNormalizati (None, None, None, 5 2048 res5a_branch2b[0][0]


activation_42 (Activation) (None, None, None, 5 0 bn5a_branch2b[0][0]


res5a_branch2c (Conv2D) (None, None, None, 2 1050624 activation_42[0][0]


res5a_branch1 (Conv2D) (None, None, None, 2 2099200 activation_40[0][0]


bn5a_branch2c (BatchNormalizati (None, None, None, 2 8192 res5a_branch2c[0][0]


bn5a_branch1 (BatchNormalizatio (None, None, None, 2 8192 res5a_branch1[0][0]


add_14 (Add) (None, None, None, 2 0 bn5a_branch2c[0][0]
bn5a_branch1[0][0]


activation_43 (Activation) (None, None, None, 2 0 add_14[0][0]


res5b_branch2a (Conv2D) (None, None, None, 5 1049088 activation_43[0][0]


bn5b_branch2a (BatchNormalizati (None, None, None, 5 2048 res5b_branch2a[0][0]


activation_44 (Activation) (None, None, None, 5 0 bn5b_branch2a[0][0]


res5b_branch2b (Conv2D) (None, None, None, 5 2359808 activation_44[0][0]


bn5b_branch2b (BatchNormalizati (None, None, None, 5 2048 res5b_branch2b[0][0]


activation_45 (Activation) (None, None, None, 5 0 bn5b_branch2b[0][0]


res5b_branch2c (Conv2D) (None, None, None, 2 1050624 activation_45[0][0]


bn5b_branch2c (BatchNormalizati (None, None, None, 2 8192 res5b_branch2c[0][0]


add_15 (Add) (None, None, None, 2 0 bn5b_branch2c[0][0]
activation_43[0][0]


activation_46 (Activation) (None, None, None, 2 0 add_15[0][0]


res5c_branch2a (Conv2D) (None, None, None, 5 1049088 activation_46[0][0]


bn5c_branch2a (BatchNormalizati (None, None, None, 5 2048 res5c_branch2a[0][0]


activation_47 (Activation) (None, None, None, 5 0 bn5c_branch2a[0][0]


res5c_branch2b (Conv2D) (None, None, None, 5 2359808 activation_47[0][0]


bn5c_branch2b (BatchNormalizati (None, None, None, 5 2048 res5c_branch2b[0][0]


activation_48 (Activation) (None, None, None, 5 0 bn5c_branch2b[0][0]


res5c_branch2c (Conv2D) (None, None, None, 2 1050624 activation_48[0][0]


bn5c_branch2c (BatchNormalizati (None, None, None, 2 8192 res5c_branch2c[0][0]


add_16 (Add) (None, None, None, 2 0 bn5c_branch2c[0][0]
activation_46[0][0]


activation_49 (Activation) (None, None, None, 2 0 add_16[0][0]


resize_1 (Lambda) (None, None, None, 2 0 activation_49[0][0]


concatenate_1 (Concatenate) (None, None, None, 3 0 resize_1[0][0]
activation_40[0][0]


conv2d_1 (Conv2D) (None, None, None, 1 393344 concatenate_1[0][0]


batch_normalization_1 (BatchNor (None, None, None, 1 512 conv2d_1[0][0]


activation_50 (Activation) (None, None, None, 1 0 batch_normalization_1[0][0]


conv2d_2 (Conv2D) (None, None, None, 1 147584 activation_50[0][0]


batch_normalization_2 (BatchNor (None, None, None, 1 512 conv2d_2[0][0]


activation_51 (Activation) (None, None, None, 1 0 batch_normalization_2[0][0]


resize_2 (Lambda) (None, None, None, 1 0 activation_51[0][0]


concatenate_2 (Concatenate) (None, None, None, 6 0 resize_2[0][0]
activation_22[0][0]


conv2d_3 (Conv2D) (None, None, None, 6 41024 concatenate_2[0][0]


batch_normalization_3 (BatchNor (None, None, None, 6 256 conv2d_3[0][0]


activation_52 (Activation) (None, None, None, 6 0 batch_normalization_3[0][0]


conv2d_4 (Conv2D) (None, None, None, 6 36928 activation_52[0][0]


batch_normalization_4 (BatchNor (None, None, None, 6 256 conv2d_4[0][0]


activation_53 (Activation) (None, None, None, 6 0 batch_normalization_4[0][0]


resize_3 (Lambda) (None, None, None, 6 0 activation_53[0][0]


zero_padding2d_1 (ZeroPadding2D (None, None, None, 2 0 activation_10[0][0]


concatenate_3 (Concatenate) (None, None, None, 3 0 resize_3[0][0]
zero_padding2d_1[0][0]


conv2d_5 (Conv2D) (None, None, None, 3 10272 concatenate_3[0][0]


batch_normalization_5 (BatchNor (None, None, None, 3 128 conv2d_5[0][0]


activation_54 (Activation) (None, None, None, 3 0 batch_normalization_5[0][0]


conv2d_6 (Conv2D) (None, None, None, 3 9248 activation_54[0][0]


batch_normalization_6 (BatchNor (None, None, None, 3 128 conv2d_6[0][0]


activation_55 (Activation) (None, None, None, 3 0 batch_normalization_6[0][0]


conv2d_7 (Conv2D) (None, None, None, 3 9248 activation_55[0][0]


batch_normalization_7 (BatchNor (None, None, None, 3 128 conv2d_7[0][0]


activation_56 (Activation) (None, None, None, 3 0 batch_normalization_7[0][0]


rbox_geo_map (Conv2D) (None, None, None, 4 132 activation_56[0][0]


rbox_angle_map (Conv2D) (None, None, None, 1 33 activation_56[0][0]


lambda_1 (Lambda) (None, None, None, 4 0 rbox_geo_map[0][0]


lambda_2 (Lambda) (None, None, None, 1 0 rbox_angle_map[0][0]


pred_score_map (Conv2D) (None, None, None, 1 33 activation_56[0][0]


pred_geo_map (Concatenate) (None, None, None, 5 0 lambda_1[0][0]
lambda_2[0][0]

Total params: 24,237,478 Trainable params: 24,183,398 Non-trainable params: 54,080


/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/engine/training.py:2087: UserWarning: Using a generator with use_multiprocessing=True and multiple workers may duplicate your data. Please consider using thekeras.utils.Sequence class. UserWarning('Using a generator with use_multiprocessing=True`' Traceback (most recent call last): Traceback (most recent call last): File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 678, in _data_generator_task self.queue.put((True, generator_output)) File "", line 2, in put Traceback (most recent call last): File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod kind, result = conn.recv() File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 678, in _data_generator_task self.queue.put((True, generator_output)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes buf = self._recv(4) File "", line 2, in put File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod kind, result = conn.recv() File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv raise EOFError File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes buf = self._recv(4) File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv raise EOFError EOFError EOFError File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 678, in _data_generator_task self.queue.put((True, generator_output)) File "", line 2, in put File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod kind, result = conn.recv() File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes buf = self._recv(4) File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv raise EOFError EOFError Process Process-8: Process Process-13: Process Process-10: Traceback (most recent call last): File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 678, in _data_generator_task self.queue.put((True, generator_output)) File "", line 2, in put File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod kind, result = conn.recv() File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes buf = self._recv(4) File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv raise EOFError EOFError Process Process-11: Traceback (most recent call last): File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 678, in _data_generator_task self.queue.put((True, generator_output)) File "", line 2, in put File "/usr/lib/python3.6/multiprocessing/managers.py", line 757, in _callmethod kind, result = conn.recv() File "/usr/lib/python3.6/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() File "/usr/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes buf = self._recv(4) File "/usr/lib/python3.6/multiprocessing/connection.py", line 383, in _recv raise EOFError EOFError Process Process-12: EOFError EOFError

During handling of the above exception, another exception occurred:

EOFError

During handling of the above exception, another exception occurred:

Traceback (most recent call last): Traceback (most recent call last):

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task self.queue.put((False, e)) File "", line 2, in put File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod conn.send((self._id, methodname, args, kwds)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes self._send(header + buf) File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task self.queue.put((False, e)) File "", line 2, in put File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod conn.send((self._id, methodname, args, kwds)) File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task self.queue.put((False, e)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes self._send(header + buf) File "", line 2, in put File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod conn.send((self._id, methodname, args, kwds)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes self._send(header + buf) BrokenPipeError: [Errno 32] Broken pipe File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe EOFError Traceback (most recent call last): EOFError

During handling of the above exception, another exception occurred:

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 676, in _data_generator_task self.queue.qsize() < self.max_queue_size): Traceback (most recent call last): File "", line 2, in qsize File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod conn.send((self._id, methodname, args, kwds)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes self._send(header + buf) File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) BrokenPipeError: [Errno 32] Broken pipe File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task self.queue.put((False, e)) File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task self.queue.put((False, e)) File "", line 2, in put File "", line 2, in put File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod conn.send((self._id, methodname, args, kwds)) File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod conn.send((self._id, methodname, args, kwds)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes self._send(header + buf) File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes self._send(header + buf) File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe BrokenPipeError: [Errno 32] Broken pipe Process Process-9: BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/johan/virtualenvs/keras2/lib/python3.6/site-packages/keras/utils/data_utils.py", line 688, in _data_generator_task self.queue.put((False, e)) File "", line 2, in put File "/usr/lib/python3.6/multiprocessing/managers.py", line 756, in _callmethod conn.send((self._id, methodname, args, kwds)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes self._send(header + buf) File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe 40 training images in /home/johan/Documents/school_projects/temp/train_data40 40 training images in /home/johan/Documents/school_projects/temp/train_data40 40 training images in /home/johan/Documents/school_projects/temp/train_data40 40 training images in /home/johan/Documents/school_projects/temp/train_data40 40 training images in /home/johan/Documents/school_projects/temp/train_data40 Epoch 1/3 40 training images in /home/johan/Documents/school_projects/temp/train_data40 Killed

jhaggle avatar Feb 14 '20 13:02 jhaggle