EfficientNet-PyTorch
EfficientNet-PyTorch copied to clipboard
lukemelas
out of memory
I use this model(efficient b4) as backbone.However , it occupies a large memory when going on training.Anyone else meets this problem? But ,in fact,it's size <100MB.
I have the same problem with almost any efficientnet model I use with a pytorch implementation. It is meant to take up less space than say a ResNet50, and in terms of number of parameters it does, but on the GPU it takes up >10GB when a ResNet50 model only takes up around 1.5GB
I'm not sure why the memory blows up but a quick fix is to reduce the batch size.
I was using efficientnet to train cifar100 which required the images to be upscaled from 32x32 to 224x224. In the beginning of my forward, I had def forward(x) : x = F.interploate(x ...) ..., but interpolate is inplace, so replacing it with just F.interploate fixed the memory problem.
I have the same issue. It seems to me that efficientnet-b4 has way more memory usage than a resnet101
Me too. According to the original paper, EfficientNet-B4 should have similar FLOPs with ResNet-50. While the practical used memory of the former is much larger than the latter. And it's not only about the memory but also the training time! I have to use 60 hours to train the same task using EfficientNet-B4, which using ResNet-50 only takes me 27 hours.
I was able to train the b0, b1, and b2 versions on a GeForce RTX 2080 Ti GPU. I receive an out of memory error when I try to train b3 and higher on the GPU. I also cannot log a b3 or higher model to TensorBoard without getting an exception. Before I spend a lot of time trying to understand whether this is an issue with my code, I want to know whether others have been able to train a b3 or higher model. I am using a batch size of 32 and resizing images according to the value returned by EfficientNet.get_image_size method.
i am recently using this implement of Efficientdet ---- https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch.I havn't met any respect cuda runtime error so far.
Same problem here, while resnet 18 takes about 1.5 gb of memory, efficientnet-b0 takes more than 4 gb!!
efficient net has about 5M parameters while resnet 18 has about 11M.
Is there sth wrong here?
Same issue here. My resnet50 takes about 2.5 GB when fine-tuning for my own dataset with batch size 2. But it blows up to 228 GB when switching to efficientnet-b7, 50 GB with efficientnet-b4. Only efficientnet-b2 fits for now.
By following this colab example, I got rid of this problem by using a resizing transformation. The size can be obtained with EfficientNet.get_image_size(model_name)
