TensorFlow-ENet icon indicating copy to clipboard operation
TensorFlow-ENet copied to clipboard

Published 20 hours ago • verified publisher icon kwotsin

the error on my personal dataset

Open kangyang94 opened this issue 6 years ago • 3 comments

I try to train the model on my own dataset which only has two types.But I got the results as below, do you know how to fix it? 2018-12-11 12:16:09.673189: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: assertion failed: [labels out of bound] [Condition x < y did not hold element-wise:x (mean_iou/confusion_matrix/control_dependency:0) = ] [0 0 0...] [y (mean_iou/ToInt64_2:0) = ] [2] [[Node: mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_INT64, DT_STRING, DT_INT64], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch/_5603, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_0, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_1, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_1/_5605, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_3, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_2/_5607)]] 2018-12-11 12:16:09.855569: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: assertion failed: [labels out of bound] [Condition x < y did not hold element-wise:x (mean_iou/confusion_matrix/control_dependency:0) = ] [0 0 0...] [y (mean_iou/ToInt64_2:0) = ] [2] [[Node: mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_INT64, DT_STRING, DT_INT64], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch/_5603, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_0, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_1, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_1/_5605, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_3, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_2/_5607)]] INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.InvalidArgumentError'>, assertion failed: [labels out of bound] [Condition x < y did not hold element-wise:x (mean_iou/confusion_matrix/control_dependency:0) = ] [0 0 0...] [y (mean_iou/ToInt64_2:0) = ] [2] [[Node: mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_INT64, DT_STRING, DT_INT64], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch/_5603, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_0, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_1, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_1/_5605, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_3, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_2/_5607)]]

Caused by op u'mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert', defined at: File "train_enet.py", line 337, in run() File "train_enet.py", line 192, in run mean_IOU, mean_IOU_update = tf.contrib.metrics.streaming_mean_iou(predictions=predictions, labels=annotations, num_classes=num_classes) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 2246, in streaming_mean_iou updates_collections=updates_collections, name=name) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/metrics_impl.py", line 944, in mean_iou num_classes, weights) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/metrics_impl.py", line 284, in _streaming_confusion_matrix labels, predictions, num_classes, weights=weights, dtype=dtypes.float64) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/confusion_matrix.py", line 176, in confusion_matrix labels, num_classes_int64, message='labels out of bound')], File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/check_ops.py", line 403, in assert_less return control_flow_ops.Assert(condition, data, summarize=summarize) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py", line 107, in wrapped return _add_should_use_warning(fn(*args, **kwargs)) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 134, in Assert condition, no_op, true_assert, name="AssertGuard") File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py", line 316, in new_func return func(*args, **kwargs) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1864, in cond orig_res_f, res_f = context_f.BuildCondBranch(false_fn) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1725, in BuildCondBranch original_result = fn() File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 132, in true_assert condition, data, summarize, name="Assert") File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_logging_ops.py", line 47, in _assert name=name) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op op_def=op_def) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1470, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): assertion failed: [labels out of bound] [Condition x < y did not hold element-wise:x (mean_iou/confusion_matrix/control_dependency:0) = ] [0 0 0...] [y (mean_iou/ToInt64_2:0) = ] [2] [[Node: mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_INT64, DT_STRING, DT_INT64], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch/_5603, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_0, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_1, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_1/_5605, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_3, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_2/_5607)]]

Traceback (most recent call last): File "train_enet.py", line 337, in run() File "train_enet.py", line 293, in run loss, training_accuracy, training_mean_IOU = train_step(sess, train_op, sv.global_step, metrics_op=metrics_op) File "train_enet.py", line 202, in train_step total_loss, global_step_count, accuracy_val, mean_IOU_val, _ = sess.run([train_op, global_step, accuracy, mean_IOU, metrics_op]) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 889, in run run_metadata_ptr) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1120, in _run feed_dict_tensor, options, run_metadata) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run options, run_metadata) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [labels out of bound] [Condition x < y did not hold element-wise:x (mean_iou/confusion_matrix/control_dependency:0) = ] [0 0 0...] [y (mean_iou/ToInt64_2:0) = ] [2] [[Node: mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_INT64, DT_STRING, DT_INT64], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch/_5603, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_0, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_1, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_1/_5605, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_3, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_2/_5607)]]

Caused by op u'mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert', defined at: File "train_enet.py", line 337, in run() File "train_enet.py", line 192, in run mean_IOU, mean_IOU_update = tf.contrib.metrics.streaming_mean_iou(predictions=predictions, labels=annotations, num_classes=num_classes) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 2246, in streaming_mean_iou updates_collections=updates_collections, name=name) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/metrics_impl.py", line 944, in mean_iou num_classes, weights) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/metrics_impl.py", line 284, in _streaming_confusion_matrix labels, predictions, num_classes, weights=weights, dtype=dtypes.float64) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/confusion_matrix.py", line 176, in confusion_matrix labels, num_classes_int64, message='labels out of bound')], File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/check_ops.py", line 403, in assert_less return control_flow_ops.Assert(condition, data, summarize=summarize) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py", line 107, in wrapped return _add_should_use_warning(fn(*args, **kwargs)) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 134, in Assert condition, no_op, true_assert, name="AssertGuard") File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py", line 316, in new_func return func(*args, **kwargs) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1864, in cond orig_res_f, res_f = context_f.BuildCondBranch(false_fn) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1725, in BuildCondBranch original_result = fn() File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 132, in true_assert condition, data, summarize, name="Assert") File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_logging_ops.py", line 47, in _assert name=name) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op op_def=op_def) File "/home/ky/miniconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1470, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): assertion failed: [labels out of bound] [Condition x < y did not hold element-wise:x (mean_iou/confusion_matrix/control_dependency:0) = ] [0 0 0...] [y (mean_iou/ToInt64_2:0) = ] [2] [[Node: mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_INT64, DT_STRING, DT_INT64], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch/_5603, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_0, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_1, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_1/_5605, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_3, mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_2/_5607)]]

kangyang94 avatar Dec 11 '18 04:12 kangyang94

Have you solved this problem? I meet the same problem, please help me

zjyLemon avatar Apr 22 '19 06:04 zjyLemon

I met the same problem.Finally I found that I have 10 classes,but I used 9 classes to train my dataset.So I got " [Condition x < y did not hold element-wise:x (mean_iou/confusion_matrix/control_dependency:0) = ] [0 0 0...] [y (mean_iou/ToInt64_2:0) = ] [2]"

x7hkvip avatar Apr 28 '19 09:04 x7hkvip

I met the same problem.Finally I found that I have 10 classes,but I used 9 classes to train my dataset.So I got " [Condition x < y did not hold element-wise:x (mean_iou/confusion_matrix/control_dependency:0) = ] [0 0 0...] [y (mean_iou/ToInt64_2:0) = ] [2]"

I met the same problem, and I changed the parameters of num_class. But it doesn't work for me. Could you tell me which parameter should I change in the codes? Thanks!

IOEvan avatar Sep 11 '19 09:09 IOEvan