Semantic-Segmentation-Suite icon indicating copy to clipboard operation
Semantic-Segmentation-Suite copied to clipboard

Error for Crop Height and Width of the FC-DenseNet103?

Open AI-ML-Enthusiast opened this issue 4 years ago • 0 comments

@GeorgeSeif Thanks for your awesome repository. I am training FCDenseNet-103 on my own dataset. It is fine with Crop_height and width=512, but when I change it to 1024, it gives the following error:

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[1,288,256,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [[node FC-DenseNet103_3/denseblock3/BatchNorm_6/FusedBatchNorm (defined at D:\feasibilityAnalysis\RiversideProject\Code\Semantic-SegmentationTF\models\FC_DenseNet_Tiramisu.py:13) ]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

[[node Mean_3 (defined at D:/feasibilityAnalysis/RiversideProject/Code/Semantic-SegmentationTF/train.py:102) ]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

My environment: Windows 10 Tensorflow 1.13 python3.6 GPU

Is there any clue to solve this issue?

Thanks in Advanced

AI-ML-Enthusiast avatar Feb 12 '20 06:02 AI-ML-Enthusiast