Error when reducing

[dsanjit]

The training went well. I am using Python 3.6 and TF 1.7.0. However when reducing the model I get the following:

2018-04-25 19:31:50.717783: W T:\src\github\tensorflow\tensorflow\core\framework\op_kernel.cc:1273] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key generator/decoder_1/deconv/filter not found in checkpoint Traceback (most recent call last): File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1327, in _do_call return fn(*args) File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1312, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1420, in _call_tf_sessionrun status, run_metadata) File "D:\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.NotFoundError: Key generator/decoder_1/deconv/filter not found in checkpoint [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]] [[Node: save/RestoreV2/_79 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_84_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "reduce_model.py", line 215, in saver.restore(sess, checkpoint) File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1775, in restore {self.saver_def.filename_tensor_name: save_path}) File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 905, in run run_metadata_ptr) File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1140, in _run feed_dict_tensor, options, run_metadata) File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1321, in _do_run run_metadata) File "D:\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1340, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.NotFoundError: Key generator/decoder_1/deconv/filter not found in checkpoint [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]] [[Node: save/RestoreV2/_79 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_84_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

Caused by op 'save/RestoreV2', defined at: File "reduce_model.py", line 213, in saver = tf.train.Saver() File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1311, in init self.build() File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1320, in build self._build(self._filename, build_save=True, build_restore=True) File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1357, in _build build_save=build_save, build_restore=build_restore) File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 809, in _build_internal restore_sequentially, reshape) File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 448, in _AddRestoreOps restore_sequentially) File "D:\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 860, in bulk_restore return io_ops.restore_v2(filename_tensor, names, slices, dtypes) File "D:\Python36\lib\site-packages\tensorflow\python\ops\gen_io_ops.py", line 1541, in restore_v2 shape_and_slices=shape_and_slices, dtypes=dtypes, name=name) File "D:\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "D:\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3290, in create_op op_def=op_def) File "D:\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1654, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

NotFoundError (see above for traceback): Key generator/decoder_1/deconv/filter not found in checkpoint [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]] [[Node: save/RestoreV2/_79 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_84_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

[gaceladri]

@dsanjit Download the pix2pix version before updated to tf 1.4 . I had the same issue and it was fixed training the model with the previous version of pix2pix since they changed the model so both models are not equal.

Hope this helps you

[CoderReece]

I had the same issue doing what @gaceladri said fix it. BUT! After making my model when i try to run the demo i get

"The name 'image_tensor:0' refers to a Tensor which does not exist. The operation, 'image_tensor', does not exist in the graph."

Did anyone else get this i can't figure out how to fix it?

Edit: the frozen model is 0 bytes in size that might be something to do with it

[DineshRajanT]

Same issue for me what @tryzombie501 says...