FastMaskRCNN unable to run this project under tf '1.2.0-rc1'.

unable to run this project under tf '1.2.0-rc1'.

Open Jiakui opened this issue 7 years ago • 12 comments

iter 108: image-id:0355947, time:0.916(sec), regular_loss: 0.202654, total-loss 1.7344(0.0361, 0.4038, 0.008075, 0.6090, 0.6774), instances: 8, batch:(21|100, 12|76, 12|12) 2017-06-16 11:52:03.372505: W tensorflow/core/framework/op_kernel.cc:1165] Invalid argument: Input to reshape is a tensor with 1 values, but the requested shape has 0 [[Node: Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](DecodeRaw_1/_555, Reshape/shape)]] iter 109: image-id:0355904, time:0.278(sec), regular_loss: 0.202653, total-loss 1.7551(0.0260, 0.4771, 0.003275, 0.5715, 0.6772), instances: 2, batch:(2|34, 4|68, 4|4) Traceback (most recent call last): File "train/train.py", line 222, in train() File "train/train.py", line 218, in train coord.join(threads) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/coordinator.py", line 389, in join six.reraise(*self._exc_info_to_raise) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/queue_runner_impl.py", line 237, in _run enqueue_callable() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1175, in _single_operation_run target_list_as_strings, status, None) File "/usr/lib/python2.7/contextlib.py", line 24, in exit self.gen.next() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status pywrap_tensorflow.TF_GetCode(status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 1 values, but the requested shape has 0 [[Node: Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](DecodeRaw_1/_555, Reshape/shape)]] [[Node: ReverseV2/_599 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_277_ReverseV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]]

Jun 16 '17 03:06 Jiakui

I can confirm that the issue exists with TF 1.2 and it will go away if you use TF 1.1 instead.

Jun 23 '17 02:06 realwecan

@realwecan I have same error. How can I downgrade tensor version to 1.1 ?

Jul 11 '17 02:07 inspire12

@inspire12 Use virtualenv and install tf1.1

Jul 11 '17 08:07 insikk

I also met this problem and downgrading TF to 1.1 solves this problem.

Not sure anyone knows why? I guess there might be some API incompatibility?

Jul 29 '17 00:07 xiaoyongzhu

What was your process for downgrading to TF 1.1? Did you have to completely reinstall FastMaskRCNN?

Jul 29 '17 00:07 AndrewKemendo

You just downgrade tensorflow version like this

ex) pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.1.0-cp36-cp36m-linux_x86_64.whl

It will work uninstall and reinstall tensorflow.

Jul 29 '17 11:07 inspire12

This worked for me by

Move MaskRCNN to another folder
Uninstall Tensorflow sudo pip uninstall tensorflow-gpu
Install Tensorflow 1.1.0 from https://pypi.python.org/pypi/tensorflow-gpu/1.1.0rc0 sudo pip install https://pypi.python.org/packages/ad/70/4ed5a455f0e64b357c844c348e3ad6bc1424ed9fc1b99e7e09d5eebffb12/tensorflow_gpu-1.1.0rc0-cp27-cp27mu-manylinux1_x86_64.whl
Move the MaskRCNN folder back into the appropriate DIR

Jul 29 '17 17:07 AndrewKemendo

Tensorflow 1.3.0 encountered the same problem. Are there anyone who solved this matter?