Run test.py to predict, show OOM error

[anxingle]

When I run test.py to predict the ISBI17 dataset(70 files), it shows the error:

Caused by op '3dconv_up4/convolution', defined at:
  File "test.py", line 141, in <module>
    predict(args)
  File "predict.py", line 45, in predict
    model = dense_rnn_net(args)
  File "/home/cec01/workspace/anxingle/cv/H-DenseUNet/hybridnet.py", line 412, in dense_rnn_net
    feature3d, classifer3d = DenseNet3D(input3d, reduction=0.5)
  File "/home/cec01/workspace/anxingle/cv/H-DenseUNet/hybridnet.py", line 172, in DenseNet3D
    conv_up4 = Conv3D(64, (3, 3, 3), padding="same", name="3dconv_up4")(up4)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/keras/engine/topology.py", line 619, in __call__
    output = self.call(inputs, **kwargs)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/keras/layers/convolutional.py", line 176, in call
    dilation_rate=self.dilation_rate)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 3567, in conv3d
    data_format=tf_data_format)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 780, in convolution
    return op(input, filter)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 868, in __call__
    return self.conv_op(inp, filter)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 520, in __call__
    return self.call(inp, filter)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 204, in __call__
    name=self.name)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1362, in conv3d
    name=name)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
    op_def=op_def)
  File "/home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
    self._traceback = tf_stack.extract_stack()

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[1,64,512,512,8] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[node 3dconv_up4/convolution (defined at /home/cec01/anaconda3/envs/tf/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3567)  = Conv3D[T=DT_FLOAT, data_format="NDHWC", dilations=[1, 1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1, 1], _device="/job:localhost/replica:0/task:0/device:GPU:0"](up_sampling3d_5/concat_2, 3dconv_up4/kernel/read)]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

	 [[{{node 2d3dclassifer/add/_9733}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_13359_2d3dclassifer/add", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

[jucyLL]

Did you solve the problem?