tensorflow-deeplab-resnet
tensorflow-deeplab-resnet copied to clipboard
DrSleep
OutofRangeError:
The checkpoint has been created. step 0 loss = 17.905, (227.301 sec/step) step 1 loss = 9.095, (221.894 sec/step) Traceback (most recent call last): File "C:\Users\Arcturus Desktop\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\client\session.py", line 1361, in _do_call return fn(*args) File "C:\Users\Arcturus Desktop\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\client\session.py", line 1340, in _run_fn target_list, status, run_metadata) File "C:\Users\Arcturus Desktop\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_1_create_inputs/batch/fifo_queue' is closed and has insufficient elements (requested 25, current size 1) [[Node: create_inputs/batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_UINT8], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](create_inputs/batch/fifo_queue, create_inputs/batch/n)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train.py", line 259, in
Caused by op 'create_inputs/batch', defined at:
File "train.py", line 259, in
OutOfRangeError (see above for traceback): FIFOQueue '_1_create_inputs/batch/fifo_queue' is closed and has insufficient elements (requested 25, current size 1) [[Node: create_inputs/batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_UINT8], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](create_inputs/batch/fifo_queue, create_inputs/batch/n)]]
We are trying to train this model on our own dataset of 90 images(for POC) and after rectifying a few errors, we are now stuck at this one. We feel that there is a problem with the batches since there is a different current size in the batch every time in every training step and that is why it stops after a few steps. But when we use get_shape() to check the batch size at each iteration it shows the correct batch size. Please let us know why this happening.
check that all the images are present; besides, since you are using windows there might be some issues with line breaks as well. So I would recommend to write a small script that verifies that all your images can be opened and read correctly.
Hey.. I quit using this repository. Instead my teammate wrote the deeplab V3 architecture completely in Kerala itself. that is giving us pretty good segmentation results. Since we are working in a private corporation I can't share that keras script with you. We might be uploading it on GitHub in some time.
Till then may be you can try writing the architecture in keras itself. V3 is pretty simple. I can guide you in writing the architecture of you wish. All the best.
Pranjal
On Sun 20 May, 2018, 2:57 PM womengjianhai, [email protected] wrote:
hi have u solve the same problem . i met it too now @tadbeer https://github.com/tadbeer
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DrSleep/tensorflow-deeplab-resnet/issues/172#issuecomment-390468600, or mute the thread https://github.com/notifications/unsubscribe-auth/AX_s77QtXSDMGU-2F1YLUrGggeK6LXdaks5t0TbzgaJpZM4S-VEA .
*we wrote the architecture in "KERAS" (not Kerela)
On Sun 20 May, 2018, 3:29 PM Pranjal Agarwal, [email protected] wrote:
Hey.. I quit using this repository. Instead my teammate wrote the deeplab V3 architecture completely in Kerala itself. that is giving us pretty good segmentation results. Since we are working in a private corporation I can't share that keras script with you. We might be uploading it on GitHub in some time.
Till then may be you can try writing the architecture in keras itself. V3 is pretty simple. I can guide you in writing the architecture of you wish. All the best.
Pranjal
On Sun 20 May, 2018, 2:57 PM womengjianhai, [email protected] wrote:
hi have u solve the same problem . i met it too now @tadbeer https://github.com/tadbeer
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DrSleep/tensorflow-deeplab-resnet/issues/172#issuecomment-390468600, or mute the thread https://github.com/notifications/unsubscribe-auth/AX_s77QtXSDMGU-2F1YLUrGggeK6LXdaks5t0TbzgaJpZM4S-VEA .
i have found the solution to it . use ' init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer()) sess.run(init_op)' to initialize the variables. Because the code forget initializing the local vairables. @tadbeer
Thanks.
On Sun 20 May, 2018, 3:34 PM womengjianhai, [email protected] wrote:
i have found the solution to it . use ' init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer()) sess.run(init_op)' to initialize the variables. Because the code forget initializing the local vairables.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DrSleep/tensorflow-deeplab-resnet/issues/172#issuecomment-390470223, or mute the thread https://github.com/notifications/unsubscribe-auth/AX_s7_5FHkXbrDLcoiwZLf532nQlB6Taks5t0T-vgaJpZM4S-VEA .
i have found the solution to it . use ' init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer()) sess.run(init_op)' to initialize the variables. Because the code forget initializing the local vairables.
)
I tried this method, but did not solve the problem
@KevinMarkVine How did you finally solve this problem? I also met the same problem as you.
