FPN_Tensorflow
FPN_Tensorflow copied to clipboard
yangxue0827
it occurs that next problem when i fix above. it seems tfrecord data problem. And first items data seems no problem. Thank you for your any help!
it occurs that next problem when i fix above. it seems tfrecord data problem. And first items data seems no problem. Thank you for your any help!
restore model 2018-07-23 20:56:06: step0 image_name:b'38bdd525-f626-4554-92ca-7ec4f95e5b2b.jpg' | rpn_loc_loss:0.2607109546661377 | rpn_cla_loss:1.163469672203064 | rpn_total_loss:1.4241806268692017 | fast_rcnn_loc_loss:0.2614363431930542 | fast_rcnn_cla_loss:0.8251640796661377 | fast_rcnn_total_loss:1.086600422859192 | total_loss:3.1513068675994873 | pre_cost_time:9.237893342971802s Traceback (most recent call last): File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call return fn(*args) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1312, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1420, in _call_tf_sessionrun status, run_metadata) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 516, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.OutOfRangeError: PaddingFIFOQueue '_1_get_batch/batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0) [[Node: get_batch/batch = QueueDequeueManyV2[component_types=[DT_STRING, DT_FLOAT, DT_INT32, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](get_batch/batch/padding_fifo_queue, get_batch/batch/n)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "train.py", line 229, in train() File "train.py", line 211, in train summary_str = sess.run(summary_op) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 905, in run run_metadata_ptr) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1140, in _run feed_dict_tensor, options, run_metadata) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run run_metadata) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.OutOfRangeError: PaddingFIFOQueue '_1_get_batch/batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0) [[Node: get_batch/batch = QueueDequeueManyV2[component_types=[DT_STRING, DT_FLOAT, DT_INT32, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](get_batch/batch/padding_fifo_queue, get_batch/batch/n)]]
Caused by op 'get_batch/batch', defined at: File "train.py", line 229, in train() File "train.py", line 34, in train is_training=True) File "../data/io/read_tfrecord.py", line 89, in next_batch dynamic_pad=True) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/training/input.py", line 989, in batch name=name) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/training/input.py", line 763, in _batch dequeued = queue.dequeue_many(batch_size, name=name) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/ops/data_flow_ops.py", line 483, in dequeue_many self._queue_ref, n=n, component_types=self._dtypes, name=name) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3476, in queue_dequeue_many_v2 component_types=component_types, timeout_ms=timeout_ms, name=name) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3290, in create_op op_def=op_def) File "/vol/venvs/tf1.7/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1654, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
OutOfRangeError (see above for traceback): PaddingFIFOQueue '_1_get_batch/batch/padding_fifo_queue' is closed and has insufficient elements (requested 1, current size 0) [[Node: get_batch/batch = QueueDequeueManyV2[component_types=[DT_STRING, DT_FLOAT, DT_INT32, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](get_batch/batch/padding_fifo_queue, get_batch/batch/n)]]
