deep_learning_tutorials icon indicating copy to clipboard operation
deep_learning_tutorials copied to clipboard

Getting issue while doing transfer learning as mentioned in S7

Open sandhyacs opened this issue 6 years ago • 0 comments

(tf) C:\Users\Admin\ssd_mobilenet\S7>python trainer.py WARNING:tensorflow:From C:\Users\Admin\ssd_mobilenet\S7\nets\mobilenet_v1.py:371: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead Loaded parameters from ./checkpoints/mobilenet_v1_1.0_224.ckpt Traceback (most recent call last): File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call return fn(*args) File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence [[{{node IteratorGetNext}} = IteratorGetNextoutput_shapes=[[?,?,?,3], [?]], output_types=[DT_FLOAT, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"]] [[{{node IteratorGetNext/_229}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_227_IteratorGetNext", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "trainer.py", line 208, in main() File "trainer.py", line 205, in main trainer.train() File "trainer.py", line 158, in train _, loss_batch = sess.run([optimizer, loss], feed_dict={training: True}) File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\client\session.py", line 929, in run run_metadata_ptr) File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run feed_dict_tensor, options, run_metadata) File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run run_metadata) File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence [[node IteratorGetNext (defined at trainer.py:115) = IteratorGetNextoutput_shapes=[[?,?,?,3], [?]], output_types=[DT_FLOAT, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"]] [[{{node IteratorGetNext/_229}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_227_IteratorGetNext", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

Caused by op 'IteratorGetNext', defined at: File "trainer.py", line 208, in main() File "trainer.py", line 205, in main trainer.train() File "trainer.py", line 115, in train images, labels = self.iterator.get_next() File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 421, in get_next name=name)), self._output_types, File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py", line 2069, in iterator_get_next output_shapes=output_shapes, name=name) File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\util\deprecation.py", line 488, in new_func return func(*args, **kwargs) File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\framework\ops.py", line 3274, in create_op op_def=op_def) File "C:\Users\Admin\AppData\Local\conda\conda\envs\tf\lib\site-packages\tensorflow\python\framework\ops.py", line 1770, in init self._traceback = tf_stack.extract_stack()

OutOfRangeError (see above for traceback): End of sequence [[node IteratorGetNext (defined at trainer.py:115) = IteratorGetNextoutput_shapes=[[?,?,?,3], [?]], output_types=[DT_FLOAT, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"]] [[{{node IteratorGetNext/_229}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_227_IteratorGetNext", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

sandhyacs avatar Jun 05 '19 10:06 sandhyacs