Learning-to-See-in-the-Dark icon indicating copy to clipboard operation
Learning-to-See-in-the-Dark copied to clipboard

Error when loading the pre-trained checkpoints

Open muepsilonpsi opened this issue 5 years ago • 2 comments

I'm trying to load the pre-trained checkpoints in train_Sony.py. The checkpoints load fine in the test_Sony and test_Fuji scripts.

This is the error. Why does the model load fine in test but not in train? The train script needs the Adam optimizer. How can this be fixed so that I can continue learning from the pre-trained checkpoints?

OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key Variable/Adam not found in checkpoint
Traceback (most recent call last):
  File "/home/my/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1278, in _do_call
    return fn(*args)
  File "/home/my/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1263, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/my/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.NotFoundError: Key Variable/Adam not found in checkpoint
	 [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/my/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1725, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/home/my/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 877, in run
    run_metadata_ptr)
  File "/home/my/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1100, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/my/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1272, in _do_run
    run_metadata)
  File "/home/my/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1291, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key Variable/Adam not found in checkpoint
	 [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

Caused by op 'save/RestoreV2', defined at:
  File "train_Sony.py", line 112, in <module>

muepsilonpsi avatar Oct 29 '18 11:10 muepsilonpsi

I came across the same error and I think it might be related to the variables initialization within the AdamOptimizer. Have you fixed it?

Melody-doudou avatar Feb 19 '19 23:02 Melody-doudou

Please I'm facing a problem when i load the pretrained model , specially when it reads the checkpoint this is the error .. how did you kindly solve it please ??

NotFoundError (see above for traceback): Key MODEL/conv7/kernel/Adam_3 not found in checkpoint [[Node: save/RestoreV2_69 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_69/tensor_names, save/RestoreV2_69/shape_and_slices)]]

BassantTolba1234 avatar Dec 21 '20 09:12 BassantTolba1234