Strange error when using my own dataset

[ebartrum]

trafficstars

Hi, I am encountering a strange error when training with my own dataset. The training works for a while then gives me this error:

` Traceback (most recent call last):
File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 12, cur rent size 0)
[[{{node shuffle_batch}} = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_STRING, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"] (shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "trainer.py", line 185, in
main()
File "trainer.py", line 182, in main trainer.train() File "trainer.py", line 113, in train self.run_single_step(self.batch_train, step=s, is_train=True) File "trainer.py", line 130, in run_single_step batch_chunk = self.session.run(batch) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run run_metadata_ptr) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run feed_dict_tensor, options, run_metadata) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run run_metadata) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 12, current size 0) [[node shuffle_batch (defined at /home/ebartrum/Documents/repos/NovelViewSynthesis-TensorFlow/input_ops.py:76) = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_STRING, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

Caused by op 'shuffle_batch', defined at: File "trainer.py", line 185, in main() File "trainer.py", line 179, in main trainer = Trainer(config, model, dataset_train, dataset_test) File "trainer.py", line 39, in init dataset, self.batch_size, is_training=True) File "/home/ebartrum/Documents/repos/NovelViewSynthesis-TensorFlow/input_ops.py", line 76, in create_input_ops min_after_dequeue=min_capacity, File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 306, in new_func return func(*args, **kwargs) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/training/input.py", line 1344, in shuffle_batch name=name) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/training/input.py", line 871, in _shuffle_batch dequeued = queue.dequeue_many(batch_size, name=name) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/ops/data_flow_ops.py", line 478, in dequeue_many self._queue_ref, n=n, component_types=self._dtypes, name=name) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3487, in queue_dequeue_many_v2 component_types=component_types, timeout_ms=timeout_ms, name=name) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func return func(*args, **kwargs) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op op_def=op_def) File "/home/ebartrum/anaconda3/envs/novel_view/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in init self._traceback = tf_stack.extract_stack()

OutOfRangeError (see above for traceback): RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 12, current size 0) [[node shuffle_batch (defined at /home/ebartrum/Documents/repos/NovelViewSynthesis-TensorFlow/input_ops.py:76) = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_STRING, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]] ` Do you know what this means?