Deep resnet runs out of memory

[weitianhan]

Hi all,

Currently I want to use Resnet as the base model for training FCN segmentation. But both Resnet-101 and Resnet-152 run out of memory on 12G Titan GPU. I am wondering how do you train this kind of very deep network. I think model parallel on multiple GPU is necessary, but I can't find any resources about this kind of implementation.

[gemfield]

You should reduce the batch size in your network.

[wangleihitcs]

Yes，I have met the same problem. My machine GPU is 1080 Ti, I use ResNet152, when batch size = 32, OOM. When batch size reduce to 8, is right!