lotto icon indicating copy to clipboard operation
lotto copied to clipboard

训练后,没法保存进度...

Open zeku2022 opened this issue 4 years ago • 0 comments

麻烦更新一下.

W tensorflow/core/framework/op_kernel.cc:1753] OP_REQUIRES failed at save_restore_v2_ops.cc:134 : Resource exhausted: checkpoints/model_checkpoint_35_temp_1b4ee8a5d4f64fbdaeb08d6891472655/part-00000-of-00001.data-00000-of-00001.tempstate4953384194912554988; No space left on device Traceback (most recent call last): File "/Users/xiangyutao/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1702, in save_v2 tld.op_callbacks, prefix, tensor_names, shape_and_slices, tensors) tensorflow.python.eager.core._FallbackException: This function does not handle the case of the path where all inputs are not already EagerTensors.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "train_and_show.py", line 64, in model.save_weights('{}/model_checkpoint_{}'.format(settings.CHECKPOINTS_PATH, epoch)) File "/Users/xiangyutao/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1167, in save_weights self._trackable_saver.save(filepath, session=session) File "/Users/xiangyutao/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/training/tracking/util.py", line 1187, in save file_prefix=file_prefix_tensor, object_graph_tensor=object_graph_tensor) File "/Users/xiangyutao/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/training/tracking/util.py", line 1135, in _save_cached_when_graph_building save_op = saver.save(file_prefix) File "/Users/xiangyutao/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/training/saving/functional_saver.py", line 256, in save sharded_saves.append(saver.save(shard_prefix)) File "/Users/xiangyutao/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/training/saving/functional_saver.py", line 73, in save return io_ops.save_v2(file_prefix, tensor_names, tensor_slices, tensors) File "/Users/xiangyutao/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1708, in save_v2 ctx=ctx) File "/Users/xiangyutao/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1730, in save_v2_eager_fallback ctx=ctx, name=name) File "/Users/xiangyutao/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute inputs, attrs, num_outputs) tensorflow.python.framework.errors_impl.ResourceExhaustedError: checkpoints/model_checkpoint(b(b((((b(((((base)

zeku2022 avatar Jul 03 '20 10:07 zeku2022